last day (15 days later) » 

20:19
2
A: Scrapy returning empty array from XPath

ᴡʜᴀᴄᴋᴀᴍᴀᴅᴏᴏᴅʟᴇ3000Add a comma in your dictionary import scrapy class AthletesSpider(scrapy.Spider): name = 'athletes' allowed_domains = ['athletic.net'] start_urls = ['https://www.athletic.net/TrackAndField/Athlete.aspx?AID=7844096#!/L0'] def parse(self, response): yield { 'a...

Oh my god, thank you that was stupid. But the second line still returns an empty array instead of the school name - is there something else missing?
One thing you could try is (if you have chrome) to inspect the page, find the element, right click, and click copy xpath. That's usually what I do to identify elements easily.
I got: //*[@id="anetMain"]/div[3]/team-nav/div/div/team-nav-logo/di‌​v/div/h1/a for the school element.
Oh, that's a helpful tip! But I'm getting another "invalid syntax" error when I tried running 'school_name' : response.xpath("//*[@id="anetMain"]/div[3]/team-nav/div/div/‌​team-nav-logo/div/‌​di‌​v/h1/a").extract_fir‌​st()
That's cause you need to put it in single quotes, not double quotes :)
'school_name' : response.xpath('//*[@id="anetMain"]/div[3]/team-nav/div/div/‌​team-nav-logo/div/‌​di‌​v/h1/a').extract_fir‌​st()
Sorry if these are stupid questions, I'm very new here - but replacing that line still returns an empty array.
20:19
Add a /text() to the xpath
Hi, I didn't want to clutter the thread with my comments anymore
but I'm still getting an empty array when I add /text()
hello?
Hi
I'm not sure what's wrong
when I try '//*[@id="anetMain"]/div[3]/team-nav/div/div/team-nav-logo/div/div/h1/a/text()'
it works for me
hmm
ok i'll try and figure it out then - thanks for your help
20:26
You could also try another library
like lxml
bs4
or maybe selenium webdriver
will do

last day (15 days later) »