Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

update GLCI (warnings + docs) #6586

Open
wants to merge 6 commits into
base: master
Choose a base branch
from
+26 −8
Open

update GLCI (warnings + docs) #6586

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
af650ad
adjust NOTE count
ben-schwen Oct 20, 2024
966c937
silent compiler warning
ben-schwen Oct 20, 2024
a2bcad6
update flags
ben-schwen Oct 20, 2024
722c0ab
remove wvla flag
ben-schwen Oct 20, 2024
956a32e
update CI docs
ben-schwen Oct 20, 2024
07029a6
update markdown highlighting
ben-schwen Oct 20, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 20 additions & 1 deletion .ci/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# data.table continuous integration and deployment

On each Pull Request opened in GitHub we run GitHub Actions test jobs to provide prompt feedback about the status of PR. Our main CI pipeline runs on GitLab CI nightly. GitLab repository automatically mirrors our GitHub repository and runs pipeline on `master` branch every night. It tests more environments and different configurations. It publish variety of artifacts.
On each Pull Request opened in GitHub we run GitHub Actions test jobs to provide prompt feedback about the status of PR. Our main CI pipeline runs on GitLab CI nightly. GitLab repository automatically mirrors our GitHub repository and runs pipeline on `master` branch every night. It tests more environments and different configurations. It publish variety of artifacts such as our [homepage](https://rdatatable.gitlab.io/data.table/) and [CRAN-like website for dev version](https://rdatatable.gitlab.io/data.table/web/packages/data.table/index.html) including windows binaries for the dev version.

## Environments

Expand Down Expand Up @@ -44,3 +44,22 @@ Base R implemented helper script, [originally proposed to base R](https://svn.r-
### [`publish.R`](./publish.R)

Base R implemented helper script to orchestrate generation of most artifacts and to arrange them nicely. It is being used only in [_integration_ stage in GitLab CI pipeline](./../.gitlab-ci.yml).

## GitLab Open Source Program

We are currently part of the GitLab for [Open Source Program](https://about.gitlab.com/solutions/open-source/). This gives us 50,000 compute minutes per month for our GitLab CI. Our license needs to renewed yearly and is currently managed by @ben-schwen.

## Updating CI pipeline

Basic CI checks are also run on every push to the GitLab repository. This can **and should** be used for PRs changing the CI pipeline before merging them to master.

```shell
# fetch changes from remote (GitHub) and push them to GitLab
git fetch [email protected]:Rdatatable/data.table.git new_branch:new_branch
git push
# after updating at GitHub, pull changes from remote and push to GitLab
git pull [email protected]:Rdatatable/data.table.git new_branch
git push
```

Make sure to include a link to the pipeline results in your PR.
9 changes: 4 additions & 5 deletions .gitlab-ci.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -140,15 +140,14 @@ test-lin-rel-cran:
<<: *test-lin
image: registry.gitlab.com/jangorecki/dockerfiles/r-base
variables:
_R_CHECK_COMPILATION_FLAGS_KNOWN_: "-Wvla"
_R_CHECK_CRAN_INCOMING_: "TRUE" ## stricter --as-cran checks should run in dev pipelines continuously (not sure what they are though)
_R_CHECK_CRAN_INCOMING_REMOTE_: "FALSE" ## Other than no URL checking (takes many minutes) or 'Days since last update 0' NOTEs needed, #3284
_R_CHECK_CRAN_INCOMING_TARBALL_THRESHOLD_: "7500000" ## bytes
_R_CHECK_PKG_SIZES_THRESHOLD_: "7" ## MB 'checking installed package size' NOTE
_R_CHECK_PKG_SIZES_THRESHOLD_: "10" ## MB 'checking installed package size' NOTE increased due to po
script:
- *install-deps
- echo 'CFLAGS=-g -O2 -fopenmp -Wall -Wvla -pedantic -fstack-protector-strong -D_FORTIFY_SOURCE=2' > ~/.R/Makevars
- echo 'CXXFLAGS=-g -O2 -fopenmp -Wall -Wvla -pedantic -fstack-protector-strong -D_FORTIFY_SOURCE=2' >> ~/.R/Makevars
- echo 'CFLAGS=-g -O2 -fopenmp -Wall -pedantic -fstack-protector-strong -D_FORTIFY_SOURCE=2' > ~/.R/Makevars
- echo 'CXXFLAGS=-g -O2 -fopenmp -Wall -pedantic -fstack-protector-strong -D_FORTIFY_SOURCE=2' >> ~/.R/Makevars
- R CMD check --as-cran $(ls -1t data.table_*.tar.gz | head -n 1)
- >-
Rscript -e 'l=tail(readLines("data.table.Rcheck/00check.log"), 1L); if (!identical(l, "Status: OK")) stop("Last line of ", shQuote("00check.log"), " is not ", shQuote("Status: OK"), " but ", shQuote(l)) else q("no")'
Expand Down Expand Up @@ -195,7 +194,7 @@ test-lin-dev-clang-cran:
- R CMD check --as-cran $(ls -1t data.table_*.tar.gz | head -n 1)
- (! grep "warning:" data.table.Rcheck/00install.out)
- >-
Rscript -e 'l=tail(readLines("data.table.Rcheck/00check.log"), 1L); notes<-"Status: 3 NOTEs"; if (!identical(l, notes)) stop("Last line of ", shQuote("00check.log"), " is not ", shQuote(notes), " (size of tarball, installed package size, non-API calls) but ", shQuote(l)) else q("no")'
Rscript -e 'l=tail(readLines("data.table.Rcheck/00check.log"), 1L); notes<-"Status: 2 NOTEs"; if (!identical(l, notes)) stop("Last line of ", shQuote("00check.log"), " is not ", shQuote(notes), " (size of tarball, non-API calls) but ", shQuote(l)) else q("no")'

# stated dependency on R
test-lin-ancient-cran:
Expand Down
2 changes: 1 addition & 1 deletion src/freadR.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@
// Where no halt is happening, we can just use raw Rprintf() or warning()
void __halt(bool warn, const char *format, ...); // see freadR.c
#define STOP(...) __halt(0, __VA_ARGS__)
static char internal_error_buff[1001]; // match internalErrSize
static char internal_error_buff[1001] __attribute__((unused)); // match internalErrSize
Copy link
Member

@MichaelChirico MichaelChirico Oct 21, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is fixable (#6468), let's mark with TODO for now if we want to ignore it in the interim.

Otherwise it's strange to the reader to see unused and then it's used in the next line.

#define INTERNAL_STOP(...) do {snprintf(internal_error_buff, 1000, __VA_ARGS__); __halt(0, "%s %s: %s. %s", _("Internal error in"), __func__, internal_error_buff, _("Please report to the data.table issues tracker"));} while (0)
#define DTPRINT Rprintf
#define DTWARN(...) warningsAreErrors ? __halt(1, __VA_ARGS__) : warning(__VA_ARGS__)
Expand Down
2 changes: 1 addition & 1 deletion src/fwrite.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@
#include "po.h"
#define STOP error
#define DTPRINT Rprintf
static char internal_error_buff[256];
static char internal_error_buff[256] __attribute__((unused));
#define INTERNAL_STOP(...) do {snprintf(internal_error_buff, 255, __VA_ARGS__); error("%s %s: %s. %s", _("Internal error in"), __func__, internal_error_buff, _("Please report to the data.table issues tracker"));} while (0)
#endif

Expand Down
Loading