You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I found a data race with reading data from Databricks simultaneously. For example, from different tables or reading catalogs metadata simultaneously.
A code to reproduce data race is
import (
"context"
"database/sql"
"fmt"
"sync"
"testing"
dbsql "github.com/databricks/databricks-sql-go"
)
func TestDatabricksDataRace(t *testing.T) {
var (
wg sync.WaitGroup
ctx = context.Background()
w = make(chan bool)
routineNum = 10
)
wg.Add(routineNum)
for i := 0; i < routineNum; i++ {
go func() {
defer wg.Done()
<-w
if err := listCatalogs(ctx); err != nil {
fmt.Println("ERROR:", err)
}
}()
}
close(w)
wg.Wait()
}
func listCatalogs(ctx context.Context) error {
connector, err := dbsql.NewConnector(
dbsql.WithServerHostname(cfg.Host),
dbsql.WithPort(int(cfg.Port)),
dbsql.WithAccessToken(cfg.AccessToken),
dbsql.WithHTTPPath(cfg.HTTPPath),
)
if err != nil {
return err
}
db := sql.OpenDB(connector)
defer db.Close()
for i := 0; i < 10; i++ {
err := func() error {
r, err := db.QueryContext(ctx, "SHOW CATALOGS;")
if err != nil {
return err
}
defer r.Close()
for r.Next() {
var s string
if err := r.Scan(&s); err != nil {
return err
}
}
return nil
}()
if err != nil {
return err
}
}
return nil
}
A command to run this test with data race detector:
go test -race -run TestDatabricksDataRace .
A root cause of data race is not initialised field loc of arrow.TimestampType. It initialised in the first call of function arrow.TimestampType :: GetZone().
A workaround of data race is:
func init() {
// init `arrow.TimestampType` before use it.
_, _ = arrow.FixedWidthTypes.Timestamp_us.(*arrow.TimestampType).GetToTimeFunc()
}
Environment:
go v1.21
databricks-sql-go v1.5.2
The text was updated successfully, but these errors were encountered:
Hello,
I found a data race with reading data from Databricks simultaneously. For example, from different tables or reading catalogs metadata simultaneously.
A code to reproduce data race is
A command to run this test with data race detector:
A root cause of data race is not initialised field loc of arrow.TimestampType. It initialised in the first call of function arrow.TimestampType :: GetZone().
A workaround of data race is:
Environment:
The text was updated successfully, but these errors were encountered: