Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

After importing dovpanda, pandas read_csv() doesn't work. #113

Open
melissadecastro opened this issue Dec 4, 2019 · 1 comment
Open
Labels
bug Something isn't working

Comments

@melissadecastro
Copy link

Brief Description

I'm trying to concatenate csv files that start with keyword(example uses A). With dovpanda, the csv files don't seem to be found. I checked the "glob(os.path.join(keyword + '*.csv'))" line by itself, that brings up a list of the correct csv files to concatenate.

System Information

Windows 10
Jupyter Notebook
Python 3.6.5 :: Anaconda, Inc.

Minimally Reproducible Code

import pandas as pd
import dovpanda

keyword = str('A')

df = pd.concat(map(pd.read_csv, glob(os.path.join(keyword + '*.csv'))))
display (df)

df.to_csv(path_or_buf=(f"Files {today}.csv"), index=False, encoding='ascii')

Error Messages

SAD PANDA

I'm so sorry, but I crashed on wrong_concat_axis hooks on concat with error descriptor 'union' of 'set' object needs an argument
But you can change that!
Please Report a bug×
Line 5: df = pd.concat(map(pd.read_csv, glob(os.path.join(keyword + '*.csv'))))
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-13-221aac6881a7> in <module>()
      3 #CRM File merge
      4 keyword = str('A')
----> 5 df = pd.concat(map(pd.read_csv, glob(os.path.join(keyword + '*.csv'))))
      6 
      7 myList = ['enrollment_id','roster_record_id','user_id','email','ssn','identify_as',

~\Anaconda3\lib\site-packages\dovpanda\base.py in run(*args, **kwargs)
    154             arguments = self._get_arguments(f, *args, **kwargs)
    155             self.run_hints(pres, arguments)
--> 156             ret = f(*args, **kwargs)
    157             self.run_hints(posts, ret, arguments)
    158             return ret

~\Anaconda3\lib\site-packages\pandas\core\reshape\concat.py in concat(objs, axis, join, join_axes, ignore_index, keys, levels, names, verify_integrity, sort, copy)
    226                        keys=keys, levels=levels, names=names,
    227                        verify_integrity=verify_integrity,
--> 228                        copy=copy, sort=sort)
    229     return op.get_result()
    230 

~\Anaconda3\lib\site-packages\pandas\core\reshape\concat.py in __init__(self, objs, axis, join, join_axes, keys, levels, names, ignore_index, verify_integrity, copy, sort)
    260 
    261         if len(objs) == 0:
--> 262             raise ValueError('No objects to concatenate')
    263 
    264         if keys is None:

ValueError: No objects to concatenate
@melissadecastro melissadecastro added the bug Something isn't working label Dec 4, 2019
@DeanLa
Copy link
Member

DeanLa commented Dec 15, 2019

Thanks @melissadecastro . Managed to reproduce.
This happened because I didn't test for generators in concat. I will work on a fix but for the meantime if you want to continue using dovpanda try:

df = pd.concat(list(map(pd.read_csv, glob(os.path.join(keyword + '*.csv')))))

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants