Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Do Not Submit] experiments #1680

Open
wants to merge 50 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
50 commits
Select commit Hold shift + click to select a range
658c952
Pipeline operations AST
wu-hui Feb 14, 2024
4ab3382
Getting ready for fluent api
wu-hui Feb 15, 2024
a3d7234
Basic fluent
wu-hui Feb 15, 2024
a837f62
Joins too
wu-hui Feb 17, 2024
c8e7854
Some fixups, and group bys
wu-hui Feb 20, 2024
c5a01a6
joins but nicer
wu-hui Feb 20, 2024
191eb44
address feedback
wu-hui Feb 22, 2024
27f2748
pagination and others
wu-hui Feb 23, 2024
55ace39
minor adds
wu-hui Feb 26, 2024
56c3a41
adding more sugar to improve brevity
wu-hui Feb 27, 2024
a5a90ab
address some feedback
wu-hui Feb 28, 2024
424ac43
support fromData
wu-hui Feb 28, 2024
4d8a205
pipeline result basic
wu-hui Mar 7, 2024
9119b95
execute
wu-hui Mar 7, 2024
3aa73de
fixing compilation error and add some high level comments
wu-hui Mar 11, 2024
b583791
setup nix for idx
wu-hui Mar 11, 2024
670ae65
Merge pull request #3 from wu-hui/wuandy/IdxSetup
wu-hui Mar 11, 2024
7216745
tweaks
wu-hui Mar 11, 2024
c46601f
pipeline result basic
wu-hui Mar 7, 2024
99e9067
execute
wu-hui Mar 7, 2024
f20e409
fixing compilation error and add some high level comments
wu-hui Mar 11, 2024
742d3c4
tweaks
wu-hui Mar 11, 2024
637d494
Merge remote-tracking branch 'pipelines/wuandy/PipelineResult' into w…
wu-hui Mar 11, 2024
7c67e99
fix errors
wu-hui Mar 11, 2024
ba521a0
fix errors
wu-hui Mar 12, 2024
6ae7b13
Add function composition example.
wu-hui Mar 12, 2024
c1825e0
build into a single jar
wu-hui Mar 13, 2024
0fe5faa
better aliasing for joins
wu-hui Mar 15, 2024
0d52712
Merge pull request #2 from wu-hui/wuandy/PipelineResult
wu-hui Mar 15, 2024
7ab3099
add stuff to support challenges
wu-hui Mar 18, 2024
57f0b74
edits
wu-hui Apr 8, 2024
0629026
Initial PP pipeline api
wu-hui Apr 9, 2024
a557d4d
pull in proto change and regenerate
wu-hui Apr 12, 2024
3369473
pull in proto change and regenerate
wu-hui Apr 12, 2024
9486027
Basic serialization
wu-hui Apr 17, 2024
ed2d276
query to pipeline
wu-hui Apr 26, 2024
f551073
e2e tests fine tuning
wu-hui Apr 29, 2024
1ceab8a
visibility changes
wu-hui May 6, 2024
df21cf2
refactoring
wu-hui May 14, 2024
cfea0fd
disable failing tests temparorily
wu-hui May 16, 2024
c2b0771
Revert "disable failing tests temparorily"
wu-hui May 16, 2024
bb16ca1
Fix missing asAlias
wu-hui May 21, 2024
424cf8b
Rename project to select
wu-hui May 21, 2024
8bc968f
fix countAll
wu-hui May 22, 2024
8c864f7
order normalization and field unify
wu-hui May 27, 2024
a79e8a7
🦉 Updates from OwlBot post-processor
gcf-owl-bot[bot] May 27, 2024
a4ea45e
Merge branch 'main' into wuandy/PplPP
wu-hui May 28, 2024
ca9a22b
Merge remote-tracking branch 'origin/wuandy/PplPP' into wuandy/PplPP
wu-hui May 28, 2024
93135a8
Add jvmfile name and string as field heuristics
wu-hui May 29, 2024
38ad69c
🦉 Updates from OwlBot post-processor
gcf-owl-bot[bot] May 29, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .idx/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@

gc/
17 changes: 17 additions & 0 deletions .idx/dev.nix
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
{ pkgs, ... }: {

# Which nixpkgs channel to use.
channel = "stable-23.11"; # or "unstable"

# Use https://search.nixos.org/packages to find packages
packages = [
pkgs.jdk11 # Or jdk8, jdk17, etc. - match your project's requirements
pkgs.maven
pkgs.kotlin
];

# Sets environment variables in the workspace
env = {
SOME_ENV_VAR = "hello";
};
}
3 changes: 3 additions & 0 deletions .vscode/settings.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
{
"java.compile.nullAnalysis.mode": "automatic"
}
8 changes: 4 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -50,20 +50,20 @@ If you are using Maven without the BOM, add this to your dependencies:
If you are using Gradle 5.x or later, add this to your dependencies:

```Groovy
implementation platform('com.google.cloud:libraries-bom:26.39.0')
implementation platform('com.google.cloud:libraries-bom:26.40.0')

implementation 'com.google.cloud:google-cloud-firestore'
```
If you are using Gradle without BOM, add this to your dependencies:

```Groovy
implementation 'com.google.cloud:google-cloud-firestore:3.21.1'
implementation 'com.google.cloud:google-cloud-firestore:3.21.3'
```

If you are using SBT, add this to your dependencies:

```Scala
libraryDependencies += "com.google.cloud" % "google-cloud-firestore" % "3.21.1"
libraryDependencies += "com.google.cloud" % "google-cloud-firestore" % "3.21.3"
```
<!-- {x-version-update-end} -->

Expand Down Expand Up @@ -222,7 +222,7 @@ Java is a registered trademark of Oracle and/or its affiliates.
[kokoro-badge-link-5]: http://storage.googleapis.com/cloud-devrel-public/java/badges/java-firestore/java11.html
[stability-image]: https://img.shields.io/badge/stability-stable-green
[maven-version-image]: https://img.shields.io/maven-central/v/com.google.cloud/google-cloud-firestore.svg
[maven-version-link]: https://central.sonatype.com/artifact/com.google.cloud/google-cloud-firestore/3.21.1
[maven-version-link]: https://central.sonatype.com/artifact/com.google.cloud/google-cloud-firestore/3.21.3
[authentication]: https://github.com/googleapis/google-cloud-java#authentication
[auth-scopes]: https://developers.google.com/identity/protocols/oauth2/scopes
[predefined-iam-roles]: https://cloud.google.com/iam/docs/understanding-roles#predefined_roles
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -486,6 +486,15 @@ public FirestoreAdminStub createStub() throws IOException {
"Transport not supported: %s", getTransportChannelProvider().getTransportName()));
}

/** Returns the endpoint set by the user or the the service's default endpoint. */
@Override
public String getEndpoint() {
if (super.getEndpoint() != null) {
return super.getEndpoint();
}
return getDefaultEndpoint();
}

/** Returns the default service name. */
@Override
public String getServiceName() {
Expand Down Expand Up @@ -1377,6 +1386,15 @@ public UnaryCallSettings.Builder<RestoreDatabaseRequest, Operation> restoreDatab
return deleteBackupScheduleSettings;
}

/** Returns the endpoint set by the user or the the service's default endpoint. */
@Override
public String getEndpoint() {
if (super.getEndpoint() != null) {
return super.getEndpoint();
}
return getDefaultEndpoint();
}

@Override
public FirestoreAdminStubSettings build() throws IOException {
return new FirestoreAdminStubSettings(this);
Expand Down
84 changes: 84 additions & 0 deletions google-cloud-firestore/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@
</parent>
<properties>
<site.installationModule>google-cloud-firestore</site.installationModule>
<kotlin.version>1.9.22</kotlin.version>
</properties>
<dependencies>
<dependency>
Expand Down Expand Up @@ -173,6 +174,17 @@
<version>3.14.0</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.jetbrains.kotlin</groupId>
<artifactId>kotlin-stdlib-jdk8</artifactId>
<version>${kotlin.version}</version>
</dependency>
<dependency>
<groupId>org.jetbrains.kotlin</groupId>
<artifactId>kotlin-test</artifactId>
<version>${kotlin.version}</version>
<scope>test</scope>
</dependency>
</dependencies>

<reporting>
Expand Down Expand Up @@ -214,6 +226,78 @@
<groupId>org.codehaus.mojo</groupId>
<artifactId>flatten-maven-plugin</artifactId>
</plugin>
<plugin>
<groupId>org.jetbrains.kotlin</groupId>
<artifactId>kotlin-maven-plugin</artifactId>
<version>${kotlin.version}</version>
<executions>
<execution>
<id>compile</id>
<phase>compile</phase>
<goals>
<goal>compile</goal>
</goals>
<configuration>
<sourceDirs>
<source>src/main/java</source>
<source>target/generated-sources/annotations</source>
</sourceDirs>
</configuration>
</execution>
<execution>
<id>test-compile</id>
<phase>test-compile</phase>
<goals>
<goal>test-compile</goal>
</goals>
<configuration>
<sourceDirs>
<source>src/test/java</source>
<source>target/generated-test-sources/test-annotations</source>
</sourceDirs>
</configuration>
</execution>
</executions>
<configuration>
<jvmTarget>1.8</jvmTarget>
</configuration>
</plugin>
<plugin>
<artifactId>maven-assembly-plugin</artifactId>
<configuration>
<descriptorRefs>
<descriptorRef>jar-with-dependencies</descriptorRef>
</descriptorRefs>
</configuration>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<executions>
<execution>
<id>default-compile</id>
<phase>none</phase>
</execution>
<execution>
<id>default-testCompile</id>
<phase>none</phase>
</execution>
<execution>
<id>compile</id>
<phase>compile</phase>
<goals>
<goal>compile</goal>
</goals>
</execution>
<execution>
<id>testCompile</id>
<phase>test-compile</phase>
<goals>
<goal>testCompile</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
</build>

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,7 @@
import com.google.api.gax.rpc.StatusCode;
import com.google.api.gax.rpc.StreamController;
import com.google.cloud.Timestamp;
import com.google.cloud.firestore.pipeline.AggregatorTarget;
import com.google.cloud.firestore.v1.FirestoreSettings;
import com.google.common.collect.ImmutableMap;
import com.google.firestore.v1.RunAggregationQueryRequest;
Expand Down Expand Up @@ -65,6 +66,16 @@ public Query getQuery() {
return query;
}

@Nonnull
public Pipeline toPipeline() {
return getQuery()
.toPipeline()
.aggregate(
this.aggregateFieldList.stream()
.map(PipelineUtils::toPipelineAggregatorTarget)
.toArray(AggregatorTarget[]::new));
}

/**
* Executes this query.
*
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,8 @@
import com.google.api.gax.rpc.StreamController;
import com.google.api.gax.rpc.UnaryCallable;
import com.google.cloud.Timestamp;
import com.google.cloud.firestore.Transaction.AsyncFunction;
import com.google.cloud.firestore.Transaction.Function;
import com.google.cloud.firestore.spi.v1.FirestoreRpc;
import com.google.common.annotations.VisibleForTesting;
import com.google.common.base.Preconditions;
Expand Down Expand Up @@ -385,30 +387,28 @@ public CollectionGroup collectionGroup(@Nonnull final String collectionId) {

@Nonnull
@Override
public <T> ApiFuture<T> runTransaction(@Nonnull final Transaction.Function<T> updateFunction) {
public <T> ApiFuture<T> runTransaction(@Nonnull final Function<T> updateFunction) {
return runAsyncTransaction(
new TransactionAsyncAdapter<>(updateFunction), TransactionOptions.create());
}

@Nonnull
@Override
public <T> ApiFuture<T> runTransaction(
@Nonnull final Transaction.Function<T> updateFunction,
@Nonnull TransactionOptions transactionOptions) {
@Nonnull final Function<T> updateFunction, @Nonnull TransactionOptions transactionOptions) {
return runAsyncTransaction(new TransactionAsyncAdapter<>(updateFunction), transactionOptions);
}

@Nonnull
@Override
public <T> ApiFuture<T> runAsyncTransaction(
@Nonnull final Transaction.AsyncFunction<T> updateFunction) {
public <T> ApiFuture<T> runAsyncTransaction(@Nonnull final AsyncFunction<T> updateFunction) {
return runAsyncTransaction(updateFunction, TransactionOptions.create());
}

@Nonnull
@Override
public <T> ApiFuture<T> runAsyncTransaction(
@Nonnull final Transaction.AsyncFunction<T> updateFunction,
@Nonnull final AsyncFunction<T> updateFunction,
@Nonnull TransactionOptions transactionOptions) {

if (transactionOptions.getReadTime() != null) {
Expand Down Expand Up @@ -519,10 +519,10 @@ public void shutdownNow() {
closed = true;
}

private static class TransactionAsyncAdapter<T> implements Transaction.AsyncFunction<T> {
private final Transaction.Function<T> syncFunction;
private static class TransactionAsyncAdapter<T> implements AsyncFunction<T> {
private final Function<T> syncFunction;

public TransactionAsyncAdapter(Transaction.Function<T> syncFunction) {
public TransactionAsyncAdapter(Function<T> syncFunction) {
this.syncFunction = syncFunction;
}

Expand Down
Loading