[Merged by Bors] - Use emptyDir for segment cache

CHANGELOG.md

@@ -8,6 +8,7 @@ All notable changes to this project will be documented in this file.
 
 - BREAKING: Support for TLS encryption (activated per default -> port changes) and TLS authentication ([#333]).
 - BREAKING: Reworked top level configuration. Deep storage, Ingestion spec, discovery config maps etc. are now subfields of `spec.clusterConfig` instead of being top level under `spec` ([#333]).
+- Use emptyDir for segment cache on historicals ([#342])
 
 ### Changed
 
@@ -21,6 +22,7 @@ All notable changes to this project will be documented in this file.
 [#333]: https://github.com/stackabletech/druid-operator/pull/333
 [#339]: https://github.com/stackabletech/druid-operator/pull/339
 [#340]: https://github.com/stackabletech/druid-operator/pull/340
+[#342]: https://github.com/stackabletech/druid-operator/pull/342
 
 ## [0.8.0] - 2022-11-07
 

deploy/config-spec/properties.yaml

@@ -736,30 +736,6 @@ properties:
           required: false
       asOfVersion: "0.0.0"
 
-  - property: &segmentCacheLocations
-      propertyNames:
-        - name: "druid.segmentCache.locations"
-          kind:
-            type: "file"
-            file: "runtime.properties"
-      datatype:
-        type: "string"
-      defaultValues:
-        - fromVersion: "0.0.0"
-          value: "[{\"path\":\"/stackable/var/druid/segment-cache\",\"maxSize\":\"300g\"}]"
-      roles:
-        - name: "broker"
-          required: false
-        - name: "coordinator"
-          required: false
-        - name: "historical"
-          required: true
-        - name: "middleManager"
-          required: false
-        - name: "router"
-          required: false
-      asOfVersion: "0.0.0"
-
   - property: &coordinatorAsOverlordOverlordService
       propertyNames:
         - name: "druid.coordinator.asOverlord.overlordService"

deploy/crd/druidcluster.crd.yaml

@@ -1147,6 +1147,63 @@ spec:
                                 type: object
                             type: object
                           storage:
+                            properties:
+                              segmentCache:
+                                default:
+                                  freePercentage: 5
+                                  emptyDir:
+                                    capacity: 1G
+                                    medium: ''
+                                properties:
+                                  emptyDir:
+                                    default:
+                                      capacity: null
+                                    properties:
+                                      capacity:
+                                        description: |-
+                                          Quantity is a fixed-point representation of a number. It provides convenient marshaling/unmarshaling in JSON and YAML, in addition to String() and AsInt64() accessors.
+
+                                          The serialization format is:
+
+                                          <quantity>        ::= <signedNumber><suffix>
+                                            (Note that <suffix> may be empty, from the "" case in <decimalSI>.)
+                                          <digit>           ::= 0 | 1 | ... | 9 <digits>          ::= <digit> | <digit><digits> <number>          ::= <digits> | <digits>.<digits> | <digits>. | .<digits> <sign>            ::= "+" | "-" <signedNumber>    ::= <number> | <sign><number> <suffix>          ::= <binarySI> | <decimalExponent> | <decimalSI> <binarySI>        ::= Ki | Mi | Gi | Ti | Pi | Ei
+                                            (International System of units; See: http://physics.nist.gov/cuu/Units/binary.html)
+                                          <decimalSI>       ::= m | "" | k | M | G | T | P | E
+                                            (Note that 1024 = 1Ki but 1000 = 1k; I didn't choose the capitalization.)
+                                          <decimalExponent> ::= "e" <signedNumber> | "E" <signedNumber>
+
+                                          No matter which of the three exponent forms is used, no quantity may represent a number greater than 2^63-1 in magnitude, nor may it have more than 3 decimal places. Numbers larger or more precise will be capped or rounded up. (E.g.: 0.1m will rounded up to 1m.) This may be extended in the future if we require larger or smaller quantities.
+
+                                          When a Quantity is parsed from a string, it will remember the type of suffix it had, and will use the same type again when it is serialized.
+
+                                          Before serializing, Quantity will be put in "canonical form". This means that Exponent/suffix will be adjusted up or down (with a corresponding increase or decrease in Mantissa) such that:
+                                            a. No precision is lost
+                                            b. No fractional digits will be emitted
+                                            c. The exponent (or suffix) is as large as possible.
+                                          The sign will be omitted unless the number is negative.
+
+                                          Examples:
+                                            1.5 will be serialized as "1500m"
+                                            1.5Gi will be serialized as "1536Mi"
+
+                                          Note that the quantity will NEVER be internally represented by a floating point number. That is the whole point of this exercise.
+
+                                          Non-canonical values will still parse as long as they are well formed, but will be re-emitted in their canonical form. (So always use canonical form, or don't diff.)
+
+                                          This format is intended to make it difficult to use these numbers without writing some sort of special handling code in the hopes that that will cause implementors to also use a fixed point implementation.
+                                        nullable: true
+                                        type: string
+                                      medium:
+                                        nullable: true
+                                        type: string
+                                    type: object
+                                  freePercentage:
+                                    format: uint16
+                                    minimum: 0.0
+                                    nullable: true
+                                    type: integer
+                                type: object
                             type: object
                         type: object
                     type: object
@@ -1293,6 +1350,63 @@ spec:
                                       type: object
                                   type: object
                                 storage:
+                                  properties:
+                                    segmentCache:
+                                      default:
+                                        freePercentage: 5
+                                        emptyDir:
+                                          capacity: 1G
+                                          medium: ''
+                                      properties:
+                                        emptyDir:
+                                          default:
+                                            capacity: null
+                                          properties:
+                                            capacity:
+                                              description: |-
+                                                Quantity is a fixed-point representation of a number. It provides convenient marshaling/unmarshaling in JSON and YAML, in addition to String() and AsInt64() accessors.
+
+                                                The serialization format is:
+
+                                                <quantity>        ::= <signedNumber><suffix>
+                                                  (Note that <suffix> may be empty, from the "" case in <decimalSI>.)
+                                                <digit>           ::= 0 | 1 | ... | 9 <digits>          ::= <digit> | <digit><digits> <number>          ::= <digits> | <digits>.<digits> | <digits>. | .<digits> <sign>            ::= "+" | "-" <signedNumber>    ::= <number> | <sign><number> <suffix>          ::= <binarySI> | <decimalExponent> | <decimalSI> <binarySI>        ::= Ki | Mi | Gi | Ti | Pi | Ei
+                                                  (International System of units; See: http://physics.nist.gov/cuu/Units/binary.html)
+                                                <decimalSI>       ::= m | "" | k | M | G | T | P | E
+                                                  (Note that 1024 = 1Ki but 1000 = 1k; I didn't choose the capitalization.)
+                                                <decimalExponent> ::= "e" <signedNumber> | "E" <signedNumber>
+
+                                                No matter which of the three exponent forms is used, no quantity may represent a number greater than 2^63-1 in magnitude, nor may it have more than 3 decimal places. Numbers larger or more precise will be capped or rounded up. (E.g.: 0.1m will rounded up to 1m.) This may be extended in the future if we require larger or smaller quantities.
+
+                                                When a Quantity is parsed from a string, it will remember the type of suffix it had, and will use the same type again when it is serialized.
+
+                                                Before serializing, Quantity will be put in "canonical form". This means that Exponent/suffix will be adjusted up or down (with a corresponding increase or decrease in Mantissa) such that:
+                                                  a. No precision is lost
+                                                  b. No fractional digits will be emitted
+                                                  c. The exponent (or suffix) is as large as possible.
+                                                The sign will be omitted unless the number is negative.
+
+                                                Examples:
+                                                  1.5 will be serialized as "1500m"
+                                                  1.5Gi will be serialized as "1536Mi"
+
+                                                Note that the quantity will NEVER be internally represented by a floating point number. That is the whole point of this exercise.
+
+                                                Non-canonical values will still parse as long as they are well formed, but will be re-emitted in their canonical form. (So always use canonical form, or don't diff.)
+
+                                                This format is intended to make it difficult to use these numbers without writing some sort of special handling code in the hopes that that will cause implementors to also use a fixed point implementation.
+                                              nullable: true
+                                              type: string
+                                            medium:
+                                              nullable: true
+                                              type: string
+                                          type: object
+                                        freePercentage:
+                                          format: uint16
+                                          minimum: 0.0
+                                          nullable: true
+                                          type: integer
+                                      type: object
                                   type: object
                               type: object
                           type: object

deploy/helm/druid-operator/configs/properties.yaml

@@ -736,30 +736,6 @@ properties:
           required: false
       asOfVersion: "0.0.0"
 
-  - property: &segmentCacheLocations
-      propertyNames:
-        - name: "druid.segmentCache.locations"
-          kind:
-            type: "file"
-            file: "runtime.properties"
-      datatype:
-        type: "string"
-      defaultValues:
-        - fromVersion: "0.0.0"
-          value: "[{\"path\":\"/stackable/var/druid/segment-cache\",\"maxSize\":\"300g\"}]"
-      roles:
-        - name: "broker"
-          required: false
-        - name: "coordinator"
-          required: false
-        - name: "historical"
-          required: true
-        - name: "middleManager"
-          required: false
-        - name: "router"
-          required: false
-      asOfVersion: "0.0.0"
-
   - property: &coordinatorAsOverlordOverlordService
       propertyNames:
         - name: "druid.coordinator.asOverlord.overlordService"