Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

How to Convert modin.pandas.DataFrame to pandas.DataFrame #896

Closed
yuanjie-ai opened this issue Dec 11, 2019 · 6 comments
Closed

How to Convert modin.pandas.DataFrame to pandas.DataFrame #896

yuanjie-ai opened this issue Dec 11, 2019 · 6 comments
Labels
question ❓ Questions about Modin

Comments

@yuanjie-ai
Copy link

How to Convert modin.pandas.DataFrame to pandas.DataFrame

@eavidan
Copy link
Collaborator

eavidan commented Dec 11, 2019

This is not part of the API as pandas.DataFrame, naturally, does not posses such a method.
You can use the private method DataFrame._to_pandas() to do this conversion.
If you would like to do this through the official API you can always save the Modin DataFrame to storage (csv, hdf, sql, ect) and then read it back using Pandas. This will probably be the safer way when working big DataFrames, to avoid out of memory issues.

@yuanjie-ai
Copy link
Author

Thanks!

@devin-petersohn devin-petersohn added the question ❓ Questions about Modin label Dec 11, 2019
@ghost
Copy link

ghost commented Feb 15, 2021

how about reading a data frame into a modin data frame ?

@devin-petersohn
Copy link
Collaborator

devin-petersohn commented Feb 15, 2021

@Vaderv if you have a pandas.DataFrame object, you can call Modin DataFrame constructor and it will be handled for you:

import pandas
import modin.pandas as pd

# have a pandas.DataFrame object called `pandas_df`
modin_df = pd.DataFrame(pandas_df)

It is as easy as that 😄. It is important to note that this will cause some data distribution.

@devin-petersohn
Copy link
Collaborator

@bdklahn Please review our code of conduct and make sure any future messages follow it: https://github.com/modin-project/modin/blob/master/CODE_OF_CONDUCT.md. Comments should be respectful and constructive.

The comment by @eavidan above answers the question. If you follow the entire thread, my comment is answering a follow-up question.

If you are running into an issue, please open a new issue. It more helpful to others looking for answers to questions to create another issue. apply is implemented, so please open a new issue if you have run into some problem. We also accept external contributions, and I'm happy to point you to the code where you can contribute a fix.

@bdklahn
Copy link

bdklahn commented Mar 27, 2021

@devin-petersohn , Sorry about that. I edited my comments accordingly.
I agree with the practice of opening a new issue, and I probably should have done that. Sorry about that, too. I also should have more carefully read the comment by @Vaderv, to note it should be a treated as separate issue.
It's probably best not to actively encourage that by answering questions about an unrelated issue, in this issue's thread. It can be confusing for people, and make it hard to track/document everything which needs to be addressed relative to a repo/issue.
It might have been best, for everyone involved, to preface your answer with something like . . . "That's a different question, and might be best addressed as a separate GitHub issue, but . . ."
Here's to hoping you will be "Gracefully accepting constructive criticism". :-)
https://github.com/modin-project/modin/blob/master/CODE_OF_CONDUCT.md

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
question ❓ Questions about Modin
Projects
None yet
Development

No branches or pull requests

5 participants