Skip to content
Toggle navigation
P
Projects
G
Groups
S
Snippets
Help
Guest User
/
general
This project
Loading...
Sign in
Toggle navigation
Go to a project
Project
Repository
Issues
0
Merge Requests
0
Pipelines
Wiki
Snippets
Settings
Activity
Graph
Charts
Create a new issue
Jobs
Commits
Issue Boards
Files
Commits
Branches
Tags
Contributors
Graph
Compare
Charts
Commit 626d62af
authored
Oct 14, 2017
by
Guest User
Browse files
Options
Browse Files
Download
Email Patches
Plain Diff
Add new file
1 parent
aaf1d7f1
Hide whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
24 additions
and
0 deletions
scrapy.py
scrapy.py
0 → 100644
View file @
626d62a
!
/
usr
/
bin
/
python
from
bs4
import
BeautifulSoup
import
requests
import
csv
import
csv
from
urllib
import
urlopen
import
re
url
=
"https://github.com/apache/incubator-mxnet"
r
=
requests
.
get
(
url
)
soup
=
BeautifulSoup
(
r
.
content
,
"html.parser"
)
Phrase
=
lambda
tag
:
(
getattr
(
tag
,
'name'
,
None
)
==
'a'
and
'href'
in
tag
.
attrs
and
'artificial-intelligence'
in
tag
.
get_text
()
.
lower
())
results
=
soup
.
find_all
(
Phrase
)
writer
=
csv
.
writer
(
open
(
"pytest.csv"
,
"wb"
))
head
=
(
"Title"
,
"URL"
)
writer
.
writerow
(
results
)
iterate
=
[]
Write
Preview
Markdown
is supported
Attach a file
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to post a comment