Grab the details pages of other websites and how to put them into our project

background:
the backend crawls the details pages of other websites. If you want to put them in our mobile phone app and keep the style uniform, you cannot use iframe. But we are not sure about the complexity of this detail page. There may be table, images or crawled web pages. What if the table of the web page is widened? Have you ever made a similar demand?
question:
where to start now, do you want to determine the input first? If so,

  1. the background returns the html text with inline style to the front end, or
  2. is returned to the front-end plain text, but where there is table or img, do you need to return it with a tag? is there a better way or script ?
Mar.08,2021

there are two ideas

  1. after grabbing in the background, parsing html classifies the data picture table, only returns the data to the front end, and the front end uses a unified layout to determine each location.
  2. after crawling, remove the style from the html, pass the html to the front end, and the front end writes the same style to the table text with css

can achieve the goal of unifying style


usually what I do is that because the page structure of the target website is different, we still need to clean the data after collection and organize the data into a structured one, so that the front end is in a fixed format, and the front end is easy to deal with. Users look at the same format if they use it.

then the trouble is that the page structure of the target site is different, each requires a separate collection program and analysis program, if the number of sites is not large, it does not matter, but after there are more sites, you need to develop the collection and analysis program into configurable, so the development may be a little more difficult.

Menu