Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

GH-41673: [Format][Docs] Add arrow format introductory page #41593

Merged
merged 65 commits into from
Oct 1, 2024
Merged

GH-41673: [Format][Docs] Add arrow format introductory page #41593

merged 65 commits into from
Oct 1, 2024

Conversation

AlenkaF
Copy link
Member

@AlenkaF AlenkaF commented May 8, 2024 •
edited
Loading

Rationale for this change

The documentation for Arrow Format could be improved:

  • all types are not listed
  • all layouts are not explained

What changes are included in this PR?

This PR includes:

  • motivation behind the columnar format
  • different physical layouts explained together with diagrams of example type in comparison to the physical layout
  • Arrow terminology
  • Extension types and sharing of Arrow data

in a separate "introduction" page with no technical details. Specifications index page is also restructured to include captions and make the left sidebar menu better organised.

Note: a table with all types listed together with their physical layout will be added in a separate PR to existing Columnar.rst page: #14752

Are these changes tested?

No, this is a docs change.

Are there any user-facing changes?

No.

@AlenkaF AlenkaF mentioned this pull request May 8, 2024
Open
@AlenkaF
Copy link
Member Author

AlenkaF commented May 9, 2024

cc @amoeba this could use a look already. I think all I wanted to add is here. Will need to do a general look through one more time before marking it ready for review though.

@AlenkaF
Copy link
Member Author

AlenkaF commented May 9, 2024

@github-actions crossbow submit preview-docs

@github-actions GitHub Actions
Copy link

github-actions bot commented May 9, 2024

Revision: 3cdd97a

Submitted crossbow builds: ursacomputing/crossbow @ actions-4a1cc2326d

Task Status
preview-docs GitHub Actions

AlenkaF
AlenkaF commented May 9, 2024
View reviewed changes
docs/source/format/index.rst Outdated
Comment on lines 40 to 34
CDataInterface
CStreamInterface
CDeviceDataInterface
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jorisvandenbossche have kept C Stream Interface in a separate file as the structure is nicer IMHO.

@github-actions github-actions bot added awaiting committer review Awaiting committer review and removed awaiting review Awaiting review labels May 9, 2024
@AlenkaF
Copy link
Member Author

AlenkaF commented May 13, 2024

@github-actions crossbow submit preview-docs

@github-actions GitHub Actions
Copy link

Revision: 4d2bf8a

Submitted crossbow builds: ursacomputing/crossbow @ actions-cc7da250f4

Task Status
preview-docs GitHub Actions

@AlenkaF
Copy link
Member Author

AlenkaF commented May 13, 2024

Not sure why the captions in the left sidebar menu are not visible in the crossbow preview build:

Screenshot 2024-05-13 at 19 47 01

but are visible for me locally:

Screenshot 2024-05-13 at 19 46 44

@AlenkaF AlenkaF marked this pull request as ready for review May 13, 2024 17:53
@AlenkaF
Copy link
Member Author

AlenkaF commented May 15, 2024

Update: I have removed the change in docs/source/format/index.rst (captions for the Specifications section) and will move it to a separate PR, see 97e4217.

@AlenkaF AlenkaF changed the title GH-39569: [Format][Docs] Add arrow format introductory page GH-41673: [Format][Docs] Add arrow format introductory page May 15, 2024
@github-actions github-actions bot added awaiting change review Awaiting change review and removed awaiting changes Awaiting changes labels Sep 25, 2024
amoeba
amoeba approved these changes Sep 25, 2024
View reviewed changes
Copy link
Member

@amoeba amoeba left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just had some minor tweaks from a final read-through. Thank you for taking this on!

docs/source/format/Intro.rst Outdated Show resolved Hide resolved
docs/source/format/Intro.rst Outdated Show resolved Hide resolved
docs/source/format/Intro.rst Outdated Show resolved Hide resolved
docs/source/format/Intro.rst Outdated Show resolved Hide resolved
felipecrv
felipecrv reviewed Sep 26, 2024
View reviewed changes
Copy link
Contributor

@felipecrv felipecrv left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I went through all the figures this time.

docs/source/format/images/bool-diagram.svg Outdated Show resolved Hide resolved
docs/source/format/images/ree-diagram.svg Outdated Show resolved Hide resolved
@github-actions github-actions bot added awaiting changes Awaiting changes and removed awaiting change review Awaiting change review labels Sep 26, 2024
@github-actions github-actions bot added awaiting change review Awaiting change review and removed awaiting changes Awaiting changes labels Sep 26, 2024
@github-actions github-actions bot added awaiting merge Awaiting merge and removed awaiting change review Awaiting change review labels Sep 26, 2024
@AlenkaF
Copy link
Member Author

AlenkaF commented Sep 26, 2024

Thank you all for reviewing this PR, not a small chunk of content!
Will keep it open for comments till Monday and then merge if there is nothing new.

raulcd
raulcd approved these changes Sep 27, 2024
View reviewed changes
Copy link
Member

@raulcd raulcd left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

just a minor grammar nit I found but looks great! Thanks @AlenkaF

docs/source/format/Intro.rst Outdated Show resolved Hide resolved
@AlenkaF
Copy link
Member Author

AlenkaF commented Sep 30, 2024

@github-actions crossbow submit preview-docs

@github-actions GitHub Actions
Copy link

Revision: 158ee32

Submitted crossbow builds: ursacomputing/crossbow @ actions-a04a6c8710

Task Status
preview-docs GitHub Actions

@AlenkaF
Copy link
Member Author

AlenkaF commented Oct 1, 2024

I am not sure why the link to the preview is not loading http://crossbow.voltrondata.com/pr_docs/41593? @assignUser is it just me?

I build the docs locally to check the html version before I merge.

@AlenkaF AlenkaF merged commit fa6a003 into apache:main Oct 1, 2024
6 checks passed
@AlenkaF AlenkaF removed the awaiting merge Awaiting merge label Oct 1, 2024
@AlenkaF AlenkaF mentioned this pull request Oct 1, 2024
Closed
@AlenkaF AlenkaF deleted the arrow-format-docs-101 branch October 1, 2024 04:32
@raulcd
Copy link
Member

raulcd commented Oct 1, 2024

I am not sure why the link to the preview is not loading http://crossbow.voltrondata.com/pr_docs/41593?

It seems to be loading for me now

@AlenkaF
Copy link
Member Author

AlenkaF commented Oct 1, 2024

Still not working for me though 🤷‍♀️

@conbench-apache-arrow conbench-apache-arrow
Copy link

After merging your PR, Conbench analyzed the 4 benchmarking runs that have been run so far on merge-commit fa6a003.

There were no benchmark performance regressions. 🎉

The full Conbench report has more details.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@paleolimbot paleolimbot paleolimbot left review comments

@amoeba amoeba amoeba approved these changes

@felipecrv felipecrv felipecrv approved these changes

@raulcd raulcd raulcd approved these changes

@jorisvandenbossche jorisvandenbossche Awaiting requested review from jorisvandenbossche

Assignees
No one assigned
Labels
Component: Documentation
Projects
None yet
Milestone
18.0.0
Development

Successfully merging this pull request may close these issues.
6 participants
@AlenkaF @amoeba @raulcd @felipecrv @jorisvandenbossche @paleolimbot