Click here to Skip to main content
15,912,457 members
Please Sign up or sign in to vote.
1.00/5 (1 vote)
See more:
hello, I have tried to scrape names of all the doctors from a website(mentioned in the code). I could get all doctors' names in the listbox. But now, I want all other details like location, consultation fees etc. of all the doctors. How to do it.Please, suggest the solution.


Note:- I can get names of all the doctors in listBox1 using the code. But I can't figure out how to scrape other details like location, consultation fees of doctors. please, suggest any hint or answer. Thanks.

What I have tried:

using System;
using System.Collections.Generic;
using System.ComponentModel;
using System.Data;
using System.Drawing;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using System.Windows.Forms;
using HtmlAgilityPack;
using System.Net;



namespace Project_1
{
    public partial class Form1 : Form
    {
        public Form1()
        {
            InitializeComponent();
            ServicePointManager.Expect100Continue = true;
            ServicePointManager.SecurityProtocol = SecurityProtocolType.Tls12;
        }

        private void button1_Click(object sender, EventArgs e)
        {
            

            
            HtmlWeb myweb = new HtmlWeb();
            int page=0;
            HtmlAgilityPack.HtmlDocument doc=null;
           
            for (page = 1; page <= 5; page++)
            {
                doc = myweb.Load("https://www.practo.com/jamnagar/doctors" + "?page=" + page);
               var headernames = (doc.DocumentNode.SelectNodes("//h2[@class='doctor-name']")).ToList();
               foreach (var item in headernames)
               {
                   listBox1.Items.Add(item.WriteContentTo());
               }

            
               
            }

    
                   
               

              

           }
        }
    }
Posted
Updated 29-Mar-22 3:22am

1 solution

We can't answer that in a "generic" manner - there is no standard way to present such information, it is down to the actual web site designer how he presents and separates such information.

So you are going to have to do it for yourself: start by using the browser to identify the data you want - in Chrome there is an "Inspect" context menu option which opens a view on the item under the mouse HTML but there is probably similar in your browser.
Then start trying to select the relevant node(s) in AgilityPack and see what you get. It make take several different tries to just get the info you need.

[edit]
Be aware that websites change - so a scrape that works today may well fail tomorrow - make your code as flexible as possible, as you will likely need to revise it many times.
[/edit]

Sorry, but we can't do that for you!
 
Share this answer
 
v2

This content, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)



CodeProject, 20 Bay Street, 11th Floor Toronto, Ontario, Canada M5J 2N8 +1 (416) 849-8900