last day (27 days later) » 

6:53 PM
1
A: How to efficiently parse large HTML div-class and span data on Python BeautifulSoup?

Life is complexHere is another answer, which can likely be consummated by someone with better soup making skills than me. I put the scraped data into 2 dictionaries, which are named balance_sheet_dict and financials_dict. I also scraped the dates associated with the columns, because I would use them in some ...

 
Zac
Great, couple of questions... 1. The first column value in operating income doesn't get displayed (i.e. 64,423,000)... 2. How to get more columns' values stored? i.e. pass a variable say 'n' and see how many columns there are, and then return all values? And then finally, 3. How to reference values, say I want ta of year 2016 - tca of year 2016... how to call the right values? Thanks again.
 
Are you referring to this tag: Operating Income or Loss and the column TTM?
 
Zac
Yes, that is the one.
 
Ok. LMK updated the answer, which will take a few minutes.
I changed the slice for chart_dates[2:] to chart_dates[1:].
You can harvest additional columns by changing the limit=int.
 
Zac
Ah I see now, okay, Reg. 3. What is the best way to reference, for example the total assets of 2016, then total current liabilities of 2016 if I have to use the values later on?
Also, reg. my 2. Is there a way to get total number of columns in the finance page? Some companies have 10 years, while some only have 3 for example. How to know how many I should fetch with limit?
 
6:53 PM
ref: What is the best way to reference, for example the total assets of 2016, then total current liabilities of 2016 if I have to use the values later on? -- are you asking how to access the dictionary key and values for 2016?
please provide an example for this - some companies have 10 years
 
Zac
Hi, go to the same link (finance.yahoo.com/quote/AAPL/financials?p=AAPL) and scroll horizontally, you can see value from TTM, then 2018 till 2008 and even more. Some companies may have only 6 years' data and so on. So, it would be great if we can get how many columns there are, so we can only fetch those specific years' data on my array. (to prevent any index out of bounds error)
 
I cannot see these extra columns, because viewing them requires premium access. So I assume that your current year parameters are 2018 to 2008, but only if these time periods exist.
I need the output of this chart_dates = date_row.find_all_next('span', limit=12)
the code that needs to be changed is this:
if date_row.text == 'Breakdown':
chart_dates = date_row.find_all_next('span', limit=12)
for dates in chart_dates[1:]:
print (dates)
Once I get this output, I will need to modify and test the code.
 
Zac
Ah yes, that needs premium and I had it. Sorry, so that's why you couldn't see that. if I add this code, it's throwing an error for me too, because we aren't logged in to premium from this Python code.
 
You have to modified the request piece to use your username and password
search for this on Google: python requests post username and password
where is the login link for premium?
 
Zac
7:27 PM
Ok, my premium may expire this year, so I am thinking of keeping the 4 values as in your code. Quick question on how to loop through the operating income values as well... Here is logic of what I want.. I want to run this module, and when run, I want the output to be:
TTM array: 365725000, 116866000, 64423000

year1 array: 375319000, 100814000, 70898000

year2 array: 321686000, 79006000, 80610000
first value of the array is TA, second is TCL and third is OIL
 
What URL is operating income values on?
and what is the title of the column?
 
Zac
Row is called "Operating Income or Loss" and column just TTM, 2018..so on
Ok, let me now try to do the python requests post username and password...
 
7:42 PM
I thought that this data was being pulled by the function get_operating_income_or_loss(soup)
and being added to this financials_dict
 
8:32 PM
I would recommend opening up another question if you need help with the python requests using post with a username and password.
 
Zac
9:15 PM
Ok, I upvoted your answer. Thanks again for the help. Yes, I will think about posting another question for the UN/PW log in to yahoo. So far, some existing answers using the requests method didn't work.
 
I would create another question.
 

  last day (27 days later) »