Extracting result URLs

Question

2.00/5 (1 vote)

See more:

How do i extract result URLs from yahoo/bing search . I dont want to use API's and i only know how to get the entire html page as the result. How do take only the result set and display by removing all other links and advertisements ??

Posted 19-Feb-14 5:06am

Member 10609453

Add a Solution

Comments

Sandip.Nascar 19-Feb-14 12:41pm

There is no other way other than manipulating the html with regex or some other way to extract actual result.
The drawbacks with this system is whenever the design layout change in future the routine to extract data will fail.

Member 10609453 27-Feb-14 10:19am

so, basically wat shud i do with this html garbage on the screen ??
https://drive.google.com/file/d/0Bzekk7dDCed8dlZRV2VENndPblU/edit?usp=sharing

Sandip.Nascar 27-Feb-14 10:50am

simple... get the html as text and use regex or other string operation to remove tags and get the actual text required.

Member 10609453 27-Feb-14 13:44pm

but the regex keeps on changing from time to time ..

Sandip.Nascar 27-Feb-14 16:59pm

why regex will change? The problem with this approach is if the html changes, your string parsing routine need a change to get the right data.

Member 10609453 27-Feb-14 13:49pm

suppose i eliminated all crap, now i have a html document in form of a notepad file ,how do i display it in a jframe ?

Sandip.Nascar 27-Feb-14 17:01pm

this is a separate question, how you pass the data to jframe.
post another question specific to your problem.

1 solution

Add a Solution

Add your solution here

Treat my content as plain text, not as HTML

Preview 0

…

Existing Members

Sign in to your account

...or Join us

Download, Vote, Comment, Publish.

Your Email
Password
Forgot your password?

Your Email
This email is in use. Do you need your password?
Optional Password

I have read and agree to the Terms of Service and Privacy Policy
Please subscribe me to the CodeProject newsletters

When answering a question please:

Read the question carefully.
Understand that English isn't everyone's first language so be lenient of bad spelling and grammar.
If a question is poorly phrased then either ask for clarification, ignore it, or edit the question and fix the problem. Insults are not welcome.
Don't tell someone to read the manual. Chances are they have and don't get it. Provide an answer or move on to the next question.

Let's work to help developers, not make them feel stupid.

This content, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)

thatraja · Answer 1 · 2014-02-19T21:48:00

Solution 1

Quote:
How do take only the result set and display by removing all other links and advertisements ??

Consider RSS[^]

Posted 19-Feb-14 21:48pm

thatraja

Comments

Member 10609453 27-Feb-14 10:17am

i need to search the websites through my app , summary wont be enuf

thatraja 27-Feb-14 10:36am

Other alternative is web scraping. In Codeproject you could find many articles on that topic.