2009年7月31日星期五

Best design for a crawling web service?

I'm developing a web service which returns the latest stock prices on a market. I've no access to the stock data. Instead I chose to scrap a HTML page to generate the data. At present my web service will do this scrapping whenever accessed. There is a performance flaw on this approach. The traffic of my hosting server shoots up for each of the service invocation. So I've two solutions to cut down the traffic.





1. Using a windows service that scraps the source HTML page in regular intervals and update a database, which will be read by the web service whenever invoked and returns the updated stock price.





2. Make the web service to update the database after scraping the HTML page, so that requests in next 10 minutes (or so) will get the data from the database, rather than scrapping the HTML page for every request.





For an user base of say 100, what will be optimal solution and why? Can you suggest a good choice for me?
Best design for a crawling web service?
It is not that easy. May be you can contact an ASP.NET expert. Check websites like http://oktutorial.com/


没有评论:

发表评论