Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

May 2021 SOPN Parsing issues #1426

Closed
symroe opened this issue Apr 8, 2021 · 51 comments
Closed

May 2021 SOPN Parsing issues #1426

symroe opened this issue Apr 8, 2021 · 51 comments
Labels
priority: medium size-large SOPN PDF parsing

Comments

@symroe
Copy link
Member

symroe commented Apr 8, 2021

Please provide a link to the PDF on the candidates site and a description of what the problem was.

Screen shots can help in some cases, but aren't always needed if the problem is obvious.
@sjorford
Copy link
Contributor

sjorford commented Apr 8, 2021 •
edited
Loading

https://candidates.democracyclub.org.uk/bulk_adding/sopn/local.stirling.forth-and-endrick.by.2021-05-06/
https://candidates.democracyclub.org.uk/elections/local.stirling.forth-and-endrick.by.2021-05-06/sopn/
Only the surname column has been parsed

@it3986

This comment has been minimized.

Sign in to view
@michaeljcollinsuk

This comment has been minimized.

Sign in to view
@JoeMitchell
Copy link
Contributor

https://candidates.democracyclub.org.uk/elections/local.leicestershire.loughborough-north-west.2021-05-06/sopn/

It's resulting in SURNAMEfirstname

@symroe
Copy link
Member Author

symroe commented Apr 8, 2021

https://candidates.democracyclub.org.uk/elections/local.fareham.fareham-east.2021-05-06/sopn/

The SOPN for Fareham Council seems to be off. It's adding "Name of ward:" and "The following people have been, or stand nominated for election to this ward. Those who no longer stand nominated have a comment in the right-hand column." as candidates, and skipping every second candidate.

@pmk01
Copy link
Contributor

pmk01 commented Apr 8, 2021

Page detection did not work on W. Oxfordshire

@illicitonion

This comment has been minimized.

Sign in to view
@illicitonion

This comment has been minimized.

Sign in to view
@illicitonion
Copy link

illicitonion commented Apr 8, 2021 •
edited
Loading

Another no-parse: https://candidates.democracyclub.org.uk/elections/local.luton.high-town.by.2021-05-06/sopn/ and https://candidates.democracyclub.org.uk/elections/local.luton.round-green.by.2021-05-06/sopn/

@illicitonion
Copy link

Nothing from https://candidates.democracyclub.org.uk/elections/local.burnley.lanehead.2021-05-06/sopn/

@illicitonion

This comment has been minimized.

Sign in to view
@jf1

This comment has been minimized.

Sign in to view
@jf1

This comment has been minimized.

Sign in to view
@jf1

This comment has been minimized.

Sign in to view
@sjorford

This comment has been minimized.

Sign in to view
@jf1

This comment has been minimized.

Sign in to view
@symroe
Copy link
Member Author

symroe commented Apr 9, 2021

Seems like none of Cornwall parsed for some reason: https://candidates.democracyclub.org.uk/elections/local.cornwall.camelford-boscastle.2021-05-06/sopn/

@illicitonion
Copy link

https://candidates.democracyclub.org.uk/elections/local.devon.yelverton-rural.2021-05-06/sopn/ identified the capital i in Judy Sara Marguerita Maciejowska's surname as an l

@jf1

This comment has been minimized.

Sign in to view
@jf1

This comment has been minimized.

Sign in to view
@jf1
Copy link

jf1 commented Apr 10, 2021 •
edited
Loading

Angela S Marynicz was parsed as Angela Marynicz S at https://candidates.democracyclub.org.uk/bulk_adding/sopn/local.devon.seaton-colyton.2021-05-06/
Also the Green candidate at https://candidates.democracyclub.org.uk/bulk_adding/sopn/local.devon.exmouth.2021-05-06/
And 4 with a middle initial at https://candidates.democracyclub.org.uk/bulk_adding/sopn/local.devon.exmouth-budleigh-salterton-coastal.2021-05-06/
1 more at https://candidates.democracyclub.org.uk/elections/local.devon.feniton-honiton.2021-05-06/sopn/
1 person with 2 initials at https://candidates.democracyclub.org.uk/bulk_adding/sopn/local.devon.sidmouth.2021-05-06/
Then nothing at all parsed for the first or last Divisions in the same pdf https://candidates.democracyclub.org.uk/bulk_adding/sopn/local.devon.broadclyst.2021-05-06/ https://candidates.democracyclub.org.uk/bulk_adding/sopn/local.devon.whimple-blackdown.2021-05-06/

3 more people with initials misplaced, in a different pdf from the same authority https://candidates.democracyclub.org.uk/bulk_adding/sopn/local.devon.axminster.2021-05-06/

@jf1

This comment has been minimized.

Sign in to view
@jf1
Copy link

jf1 commented Apr 10, 2021

3 of 5 Divisions were fine in https://candidates.democracyclub.org.uk/upload_document/posts_for_document/21870/
But nothing at https://candidates.democracyclub.org.uk/bulk_adding/sopn/local.devon.ashburton-buckfastleigh.2021-05-06/ and https://candidates.democracyclub.org.uk/bulk_adding/sopn/local.devon.bideford-east.2021-05-06/

@jf1

This comment has been minimized.

Sign in to view
@jf1
Copy link

jf1 commented Apr 11, 2021

From RyanC... Another example of bad parsing https://candidates.democracyclub.org.uk/elections/local.leicestershire.loughborough-east.2021-05-06/sopn/
image

@jf1
Copy link

jf1 commented Apr 11, 2021 •
edited
Loading

SUCCESS from a HTML->PDF converted SoPN https://candidates.democracyclub.org.uk/elections/local.south-hams.ivybridge-west.by.2021-05-06/sopn/
Well, not really success as I uploaded a results file instead of a SoPN, but the bot parsed it all ok so I'll leave the above for reference.

@jf1
Copy link

jf1 commented Apr 11, 2021 •
edited
Loading

Didn't get any forenames from https://candidates.democracyclub.org.uk/bulk_adding/sopn/local.welwyn-hatfield.brookmans-park-little-heath.2021-05-06/
Same here (via JoeCL) https://candidates.democracyclub.org.uk/elections/local.welwyn-hatfield.sherrards.2021-05-06/sopn/
And (via BoP) https://candidates.democracyclub.org.uk/elections/local.welwyn-hatfield.haldens.2021-05-06/sopn/

@jf1
Copy link

jf1 commented Apr 11, 2021

Via BoP: https://candidates.democracyclub.org.uk/elections/local.suffolk.thedwastre-north.2021-05-06/sopn/ parsed Harry Richardson as Richardson Harry

@symroe symroe mentioned this issue Apr 26, 2021
Closed
@VirginiaDooley
Copy link
Contributor

VirginiaDooley commented Jan 27, 2022 •
edited
Loading

Bot consistently isn't picking up the Party for "Labour and Co-operative Party" candidates in the Westcountry. Candidate name is showing correctly in the pre-filled data from the bot but the Party field is just blank

Examples - https://candidates.democracyclub.org.uk/elections/local.gloucestershire.dursley.2021-05-06/sopn/ https://candidates.democracyclub.org.uk/elections/local.gloucestershire.bisley-and-painswick.2021-05-06/sopn/ https://candidates.democracyclub.org.uk/elections/local.gloucestershire.cam-valley.2021-05-06/sopn/

@EdwardBetts This bug has been fixed with #1711

@VirginiaDooley
Copy link
Contributor

https://candidates.democracyclub.org.uk/elections/local.fareham.fareham-east.2021-05-06/sopn/

The SOPN for Fareham Council seems to be off. It's adding "Name of ward:" and "The following people have been, or stand nominated for election to this ward. Those who no longer stand nominated have a comment in the right-hand column." as candidates, and skipping every second candidate.

There may be some issue with the version of this PDF. It uploads blank when I test it.

@VirginiaDooley
Copy link
Contributor

VirginiaDooley commented Jan 27, 2022 •
edited
Loading

@VirginiaDooley
Copy link
Contributor

Page detection did not work on W. Oxfordshire

@pmk01 Do you have a ballot id I can test?

@pmk01
Copy link
Contributor

pmk01 commented Jan 28, 2022

I honestly don't remember this! It's possible this was raised by a volunteer in the Slack and I added it here. I presume this is the issue: https://candidates.democracyclub.org.uk/elections/local.west-oxfordshire.witney-east.2021-05-06/sopn/

@VirginiaDooley
Copy link
Contributor

https://candidates.democracyclub.org.uk/elections/local.leeds.burmantofts-richmond-hill.2021-05-06/sopn/ - did not parse out any values, and whatever fetched the PDF messed up the pages such that PETERS, Karen was missing.

@illicitonion This SOPN was either incorrectly published by the council or possibly printed to PDF from an html file (or both).

@VirginiaDooley
Copy link
Contributor

https://candidates.democracyclub.org.uk/elections/local.derbyshire.derwent-valley.2021-05-06/sopn/ - did not parse out any values, nothing looks particularly strange about the PDF

@illicitonion This was solved with #1697

@VirginiaDooley
Copy link
Contributor

Another no-parse: https://candidates.democracyclub.org.uk/elections/local.luton.high-town.by.2021-05-06/sopn/ and https://candidates.democracyclub.org.uk/elections/local.luton.round-green.by.2021-05-06/sopn/

@illicitonion Table parsing error; Header cannot be found; moving to #1728

@VirginiaDooley
Copy link
Contributor

elections/local.burnley.lanehead.2021-05-06/sopn/

Nothing from https://candidates.democracyclub.org.uk/elections/local.burnley.lanehead.2021-05-06/sopn/

@illicitonion Ward matching error (table parsing error); moving to #1728

@VirginiaDooley
Copy link
Contributor

Nothing from https://candidates.democracyclub.org.uk/elections/local.sunderland.millfield.2021-05-06/sopn/

@illicitonion Fixed with #1711

@VirginiaDooley
Copy link
Contributor

Trivial but maybe worth fixing - from a blank line on the pdf, an Independent candidate with no name was proposed. https://candidates.democracyclub.org.uk/bulk_adding/sopn/local.cambridgeshire.sutton.2021-05-06/

@jf1 Table parsing error; moving to #1728

@VirginiaDooley
Copy link
Contributor

VirginiaDooley commented Jan 31, 2022 •
edited
Loading

elections/local.lewisham.bellingham.by.2021-05-06/sopn/

Lewisham People Before Profit doesn't show on the bulk add page, even after clicking "Load more parties" image https://candidates.democracyclub.org.uk/elections/local.lewisham.bellingham.by.2021-05-06/sopn/

It does show if you add the candidate directly to the ballot page image

@sjorford Fixed with #1712; Further improvements in #1730

@VirginiaDooley
Copy link
Contributor

VirginiaDooley commented Jan 31, 2022 •
edited
Loading

https://candidates.democracyclub.org.uk/bulk_adding/sopn/local.liverpool.mossley-hill.2021-05-06/?edit=1 proposed Green Party for The Liberal Party Candidate Then when I reloaded that page all of the proposed parties were Green Party.

@jf1 Fixed with #1711

@VirginiaDooley
Copy link
Contributor

VirginiaDooley commented Jan 31, 2022 •
edited
Loading

Seems like none of Cornwall parsed for some reason: https://candidates.democracyclub.org.uk/elections/local.cornwall.camelford-boscastle.2021-05-06/sopn/

@symroe Table parsing error: No ParsedSOPN; moving to #1728

@VirginiaDooley
Copy link
Contributor

VirginiaDooley commented Jan 31, 2022 •
edited
Loading

https://candidates.democracyclub.org.uk/elections/local.devon.yelverton-rural.2021-05-06/sopn/ identified the capital i in Judy Sara Marguerita Maciejowska's surname as an l

@illicitonion Table parsing error; moving to #1728

@VirginiaDooley
Copy link
Contributor

The parser didn't get the description Christchurch Independents (which is in the Click to load more... section) https://candidates.democracyclub.org.uk/bulk_adding/sopn/local.bournemouth-christchurch-and-poole.commons.by.2021-05-06/

@jf1 Fixed with #1712; Further improvements in #1730

@VirginiaDooley
Copy link
Contributor

@jf1
Angela S Maryniczwas parsed asAngela Marynicz S` at https://candidates.democracyclub.org.uk/bulk_adding/sopn/local.devon.seaton-colyton.2021-05-06/

Table parsing error; moving to #1728

Also the Green candidate at https://candidates.democracyclub.org.uk/bulk_adding/sopn/local.devon.exmouth.2021-05-06/

Fixed with #1712

And 4 with a middle initial at https://candidates.democracyclub.org.uk/bulk_adding/sopn/local.devon.exmouth-budleigh-salterton-coastal.2021-05-06/

Table parsing error; moving to #1728

1 more at https://candidates.democracyclub.org.uk/elections/local.devon.feniton-honiton.2021-05-06/sopn/

Table parsing error; moving to #1728

1 person with 2 initials at https://candidates.democracyclub.org.uk/bulk_adding/sopn/local.devon.sidmouth.2021-05-06/

Table parsing error; moving to #1728

Then nothing at all parsed for the first or last Divisions in the same pdf https://candidates.democracyclub.org.uk/bulk_adding/sopn/local.devon.broadclyst.2021-05-06/ https://candidates.democracyclub.org.uk/bulk_adding/sopn/local.devon.whimple-blackdown.2021-05-06/

Fixed with #1712; Further improvements in #1730 but still has initial placing issue in table parsing so will move to #1728

3 more people with initials misplaced, in a different pdf from the same authority https://candidates.democracyclub.org.uk/bulk_adding/sopn/local.devon.axminster.2021-05-06/

Table parsing error; moving to #1728

@VirginiaDooley
Copy link
Contributor

I honestly don't remember this! It's possible this was raised by a volunteer in the Slack and I added it here. I presume this is the issue: https://candidates.democracyclub.org.uk/elections/local.west-oxfordshire.witney-east.2021-05-06/sopn/

@pmk01 Page matching error; moving to #1726

@VirginiaDooley VirginiaDooley mentioned this issue Jan 31, 2022
Open
@VirginiaDooley
Copy link
Contributor

From RyanC... Another example of bad parsing https://candidates.democracyclub.org.uk/elections/local.leicestershire.loughborough-east.2021-05-06/sopn/ image

Name parsing error in table parsing; moving to #1728 for further inspection

@VirginiaDooley
Copy link
Contributor

Didn't get any forenames from https://candidates.democracyclub.org.uk/bulk_adding/sopn/local.welwyn-hatfield.brookmans-park-little-heath.2021-05-06/ Same here (via JoeCL) https://candidates.democracyclub.org.uk/elections/local.welwyn-hatfield.sherrards.2021-05-06/sopn/ And (via BoP) https://candidates.democracyclub.org.uk/elections/local.welwyn-hatfield.haldens.2021-05-06/sopn/

Table extraction error; moving to #1727

@VirginiaDooley VirginiaDooley mentioned this issue Jan 31, 2022
Open
@VirginiaDooley
Copy link
Contributor

Via BoP: https://candidates.democracyclub.org.uk/elections/local.suffolk.thedwastre-north.2021-05-06/sopn/ parsed Harry Richardson as Richardson Harry

Table parsing error; moving to #1728

@VirginiaDooley
Copy link
Contributor

VirginiaDooley commented Jan 31, 2022 •
edited
Loading

Trivial but maybe worth fixing - from a blank line on the pdf, an Independent candidate with no name was proposed. https://candidates.democracyclub.org.uk/bulk_adding/sopn/local.cambridgeshire.sutton.2021-05-06/

@jf1 Fixed with #1731 (review)

@VirginiaDooley VirginiaDooley mentioned this issue Feb 14, 2022
Merged
@symroe
Copy link
Member Author

symroe commented Oct 26, 2022

Closing this as we've either fixed the issues or we're tracking new issues in #1727

@symroe symroe closed this as completed Oct 26, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
priority: medium size-large SOPN PDF parsing
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
9 participants
@symroe @illicitonion @jf1 @it3986 @VirginiaDooley @JoeMitchell @sjorford @michaeljcollinsuk @pmk01