greptilian logo

IRC log for #sourcefu, 2019-07-24

http://sourcefu.com

| Channels | #sourcefu index | Today | | Search | Google Search | Plain-Text | plain, newest first | summary

All times shown according to UTC.

Time S Nick Message
21:08 kzisme prologic: How did the wikipedia crawl go?
21:16 prologic Oh I'm still working on it
21:16 prologic But I was able to crawl ~50M articles in ~5-6hrs
21:17 prologic right nw I'm just trying to fit  all the pieces together into something  that half works, crawls, indexes and has a nice web ui
21:17 prologic but also that the extraction process  extracts meaningful text (hard to get right)
22:07 pdurbin Does Wikipedia have an API?

| Channels | #sourcefu index | Today | | Search | Google Search | Plain-Text | plain, newest first | summary

http://sourcefu.com