Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Fix curate internal quotes take 2 #1565

Open
wants to merge 2 commits into
base: master
Choose a base branch
from
Open

Fix curate internal quotes take 2 #1565

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
8b00fbd
Revert "write_records_to_tsv: Stop quoting output TSV"
joverlee521 Aug 1, 2024
ad029f6
curate: fix endless additional quotes
joverlee521 Aug 1, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 3 additions & 4 deletions augur/io/metadata.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -273,7 +273,8 @@ def visible_worksheet(s: calamine.SheetMetadata) -> bool:
# change in a future Python version.
raise InvalidDelimiter from error

metadata_reader = csv.DictReader(handle, dialect=dialect)
# Only use the dialect delimiter and keep all other default format params
metadata_reader = csv.DictReader(handle, delimiter=dialect.delimiter)

columns, records = metadata_reader.fieldnames, iter(metadata_reader)

Expand Down Expand Up @@ -548,9 +549,7 @@ def write_records_to_tsv(records, output_file):
output_columns,
extrasaction='ignore',
delimiter='\t',
lineterminator='\n',
quoting=csv.QUOTE_NONE,
quotechar=None,
lineterminator='\n'
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

very nit, much pick

Suggested change
lineterminator='\n'
lineterminator='\n',

)
tsv_writer.writeheader()
tsv_writer.writerow(first_record)
Expand Down
6 changes: 3 additions & 3 deletions tests/functional/curate/cram/metadata-output-with-internal-quotes.t
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,18 +12,18 @@ Create NDJSON with internal quotes
> ~~

Test passthru with output to TSV.
This should not add any quotes around the field with internal quotes.
This should add double quotes around the internal quotes to match CSV-like quoting.

$ cat records.ndjson \
> | ${AUGUR} curate passthru \
> --output-metadata output-metadata.tsv

$ cat output-metadata.tsv
strain\tsubmitting_lab (esc)
sequence_A\tSRC VB "Vector", Molecular Biology of Genomes (esc)
sequence_A\t"SRC VB ""Vector"", Molecular Biology of Genomes" (esc)

Run the output TSV through augur curate passthru again.
The new output should still be identical to the first output.
The new output should still be identical to the first output because it is already double quoted.

$ ${AUGUR} curate passthru \
> --metadata output-metadata.tsv \
Expand Down
2 changes: 1 addition & 1 deletion tests/io/test_metadata.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -458,7 +458,7 @@ def output_records():
def expected_output_tsv():
return (
"strain\tcountry\tdate\n"
'SEQ_A\t"USA"\t2020-10-01\n'
'SEQ_A\t"""USA"""\t2020-10-01\n'
"SEQ_T\tUSA\t2020-10-02\n"
)

Expand Down
Loading