Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add script for scraping Smogon movesets #363

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

Devin-Mix
Copy link

@Devin-Mix Devin-Mix commented Jan 26, 2022

This script adds functionality to the repository by providing a means to scrape competitive Pokemon data from Smogon. This data is subsequently saved to smogon-movesets.csv in the folder /pokedex/data/csv. The data itself includes a header line, which outlines the following broad categories which the data covers. For each moveset of each Pokemon in each generation, the following data are available:

  • name
  • gen
  • format
  • overview
  • comments
  • set name
  • pokemon
  • shiny
  • gender
  • levels
  • description
  • abilities
  • items
  • move 1
  • move 2
  • move 3
  • move 4
  • ev configs
  • iv configs
  • natures
  • writing teams
  • Written by

Data scraping takes me around eight minutes on my network, but your mileage may vary.

I would include a copy of the data directly, but the CSV file produced contains 5232 lines and is over 100mb, which is GitHub's limit for customers without Large Storage packages. A copy is thus available here.

The script itself is intended to be run once per user in an interactive environment, but could easily be modified to not include prints or a user confirmation prompt if desired.

I'm unsure how this data could see use in the remainder of the utility offered by this project, but it opens the door to future CLI engagements with Smogon's data by abstracting the process of fetching and cleaning their JSON.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant