[µo°Ý] How to scrape this website dynamic data from Javascript with VBA? VBA, data scraping, javascript

DanielWONG

Website: https://www.hangseng.com/en-hk/personal/insurance-mpf/e-mpf/fund-price-performance/

Targets of automate data scraping: fund price as highlighted in the below figure

¤U¸ü (111.61 KB)
target data

2022-11-19 16:41

Tools: VBA(Excel), Chrome driven by Selenium

The code started as below:

1. Chrome.ExecuteScript "window.open('https://www.hangseng.com/en-hk/e-services/e-mpf/fund-price- performance/price/','_blank');"
   
2. 
   
3. Chrome.SwitchToNextWindow
   
4. 
   
5. Chrome.FindElementByXPath("//a[text()='Acknowledge']").Click

½Æ»s¥N½X

Then I have no further idea to retrieve those specified data. Please provide help. Thanks a lot!

DanielWONG

Thanks a lot! I found if  I  switch to this OCR site https://ocr.space/
The code should works well.
Respect for your nice work!

¦^´_ 6# singo1232001

singo1232001

¦^´_ 3# DanielWONG


    «Ü¶§¬Kªºª±ªk ·|¦³¤@©w¾÷²v¥¢®Ä

  ¥þ¬ÝOCR ºô¯¸ªº®ÄªG

singo1232001

¦^´_ 3# DanielWONG

Ãö©ó²Ä¤G­Ó¹Ï¤ù¿ëÃѪº°ÝÃD
ºô¸ô¤W¦³¬d¨ìªº¸ê®Æ

¦³¨âºØ°µªk
1.ª½±µ§ï¥Îpython ½Õ¥Î¹Ï§Î¿ëÃѪº¥\¯à (¸û²³æ)
¬Æ¦Üyt¦³¼v¤ùª½±µ±Ð§A
¹Ï«¬¿ëÃÑ ¸ò ª¦Â㣦³   ª¦ÂΤ@¼Ë¥Îselenium


2.¥Îvba ª¦ÂΧì¹Ï>¦b¤W¶Ç¨ì¦Ê«×¹Ï¤ù¿ëÃѪºapi §ä¨ì§A­nªº¨º¬q
¦ý¯à§_¨Ï¥Î ©|¥¼¹ê»Ú°µ¹L  ­n½Ð°ª¤â

3.vba(shell)+python
¤]¤j­P¦p¤W §â1¸ò2¦X°_¨Óª±¥X·sªá¼Ë ¸û½ÆÂø

¬Ý¦³¨S¦³°ª¤â¤j¤j·|
©Î½Ð°ª¤âÀ°¦£
©Î§ä²^x

singo1232001

¦^´_ 3# DanielWONG


    tag <a>ªº¤¸¯À ¦L¶H¦³2000­Ó¥H¤W

¦Ó§Ú­Ì­nªº¤j¬ù¦b²Ä1275­Ó  

±q1¶}©l¶] ­n¶]«Ü¤[ ¤~¨ì1275 ¤j·§10~20¬í(²¦³º¬O¨C­Óª«¥ó¥´¶}Àˬd·|¤[)

¥»¨Ó¥´ºâª½±µ¥Î A_s(1275).click ³Ì§Ö

¦ý¤S©Èºô¯¸ ¥i¯à1~1274¤¤  ¬Y¨Ç¤¸¯À¦³®É­Ô®ø¥¢ ©ÎªÌ¼W¥[

³y¦¨1275ªº¦ì¸m·|ÅܤÆ

©Ò¥H§ì­Ó®e³\­È ±q1100Ä~Äò§ä «e­±ªº1~1099 ³£ª½±µ©¿²¤  °£«Dºô­¶ÅܤƧï«Ü¤j ¤Ö¤@°ï¤¸¯À

³o¼Ë¤£·|¤ÓºC ¤]¦³®e¿ùªÅ¶¡

DanielWONG

¦^´_ 2# singo1232001


  The code is very simple , but very productive!
There are also two questions:
1)

1. Set A_s = chrome.FindElementsBytag("a")
   
2.     For i = 1100 To A_s.Count

½Æ»s¥N½X

Only 139 "<a" tag can be searched in the website source code, what is the meaning of 1100 here?

2)  for another website http://www.aastocks.com/tc/stocks/quote/detailchart.aspx?symbol=110000
It is possible to extract the highlighted RSI data ?

¤U¸ü (69.87 KB)

2022-11-20 10:45

Thank you very much!

singo1232001

¦^´_ 1# DanielWONG


Sub test2()
Set chrome = CreateObject("selenium.chromedriver")
chrome.get "https://www.hangseng.com/en-hk/personal/insurance-mpf/e-mpf/fund-price-performance/"
chrome.Wait 1000

Set A_s = chrome.FindElementsBytag("a")
    For i = 1100 To A_s.Count
     If A_s(i).Text = "Acknowledge" Then A_s(i).Click: Exit For
    Next


ReDim ar(1 To 1000, 1 To 1)
Set td_s = chrome.FindElementsBytag("td")
    For Each Z In td_s
        If Z.attribute("headers") = "price_header1" Then w = w + 1: ar(w, 1) = Z.Text
    Next
   
Cells.ClearContents
Cells(1, 1).Resize(w, 1) = ar
End Sub