Skip to content

Commit

Permalink
GH-37337: [MATLAB] Add arrow.array.Time64Array class (#37368)
Browse files Browse the repository at this point in the history
### Rationale for this change

Now that `arrow.type.Time64Type` class has been added to the MATLAB Interface (#37287), we can add the `arrow.array.Time64Array` class.

### What changes are included in this PR?

1. New `arrow.array.Time64Array` class.
2. New `arrow.type.traits.Time64Traits` class.
3. Enhanced `arrow.array(data)` to return a `arrow.array.Time64Array` instance if the input `data` is a MATLAB `duration` array. The MATLAB `duration` type represents length of time in fixed-length units.
4. Enhanced `arrow.type.traits.traits(id)` to return an `arrow.type.traits.Time64Traits` instance if `id` is an `arrow.type.ID.Time64` value or equal to the string `"duration".`
5. Updated the test utility `arrow.internal.test.tabular.createAllSupportedArrayTypes()` to include an `arrow.array.Time64Array` instance in its output.
6. Updated the test utility `arrow.internal.test.tabular.createTableWithSupportedTypes()` to include a `duration` variable on the returned `table`. 
7. Updated the README.md to include `Time32Array` and `Time64Array`.

**Example Usage**
```matlab

% Create a duration array
>> ms = milliseconds([100 150 1001])

ms = 

  1×3 duration array

     0.1 sec    0.15 sec   1.001 sec

% Create an arrow.type.Time64Array
>> time = arrow.array(ms)

time = 

[
  00:00:00.100000,
  00:00:00.150000,
  00:00:01.001000
]

% Create a duration array from an arrow.array.Time64Array
>> data = duration(time) 

data = 

  3×1 duration array

     0.1 sec
    0.15 sec
   1.001 sec

```

### Are these changes tested?

Yes.

1. Added a new test class `tTime64Array.m`.
2. Added a new test class `tTime64Traits.m`
3. Added new test cases to `tArray.m` and `ttraits.m`

### Are there any user-facing changes?

Yes. Users can now create `arrow.array.Time64Array`s from  MATLAB `duration`s

* Closes: #37337

Authored-by: Sarah Gilmore <[email protected]>
Signed-off-by: Kevin Gurney <[email protected]>
  • Loading branch information
sgilmore10 authored Aug 25, 2023
1 parent fa0af70 commit faccbef
Show file tree
Hide file tree
Showing 28 changed files with 667 additions and 111 deletions.
2 changes: 2 additions & 0 deletions matlab/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -51,6 +51,8 @@ Supported `arrow.array.Array` types are included in the table below.
| `logical` | `BooleanArray` |
| `string` | `StringArray` |
| `datetime` | `TimestampArray` |
| `duration` | `Time32Array` |
| `duration` | `Time64Array` |

## Prerequisites

Expand Down
38 changes: 2 additions & 36 deletions matlab/src/cpp/arrow/matlab/array/proxy/time32_array.cc
Original file line number Diff line number Diff line change
Expand Up @@ -16,47 +16,13 @@
// under the License.

#include "arrow/matlab/array/proxy/time32_array.h"

#include "arrow/matlab/type/time_unit.h"
#include "arrow/util/utf8.h"
#include "arrow/matlab/array/proxy/time_array.h"

namespace arrow::matlab::array::proxy {

// Specialization of NumericArray::Make for arrow::Time32Type
template <>
libmexclass::proxy::MakeResult Time32Array::make(const libmexclass::proxy::FunctionArguments& constructor_arguments) {
namespace mda = ::matlab::data;
using MatlabBuffer = arrow::matlab::buffer::MatlabBuffer;
using Time32Array = arrow::Time32Array;
using Time32ArrayProxy = arrow::matlab::array::proxy::NumericArray<arrow::Time32Type>;

mda::StructArray opts = constructor_arguments[0];

// Get the mxArray from constructor arguments
const mda::TypedArray<int32_t> time32_mda = opts[0]["MatlabArray"];
const mda::TypedArray<bool> validity_bitmap_mda = opts[0]["Valid"];

const mda::TypedArray<mda::MATLABString> units_mda = opts[0]["TimeUnit"];

// extract the time unit
const std::u16string& u16_timeunit = units_mda[0];
MATLAB_ASSIGN_OR_ERROR(const auto time_unit,
arrow::matlab::type::timeUnitFromString(u16_timeunit),
error::UKNOWN_TIME_UNIT_ERROR_ID)

// create the Time32Type
auto data_type = arrow::time32(time_unit);
auto array_length = static_cast<int32_t>(time32_mda.getNumberOfElements()); // cast size_t to int32_t

auto data_buffer = std::make_shared<MatlabBuffer>(time32_mda);

// Pack the validity bitmap values.
MATLAB_ASSIGN_OR_ERROR(auto packed_validity_bitmap,
bit::packValid(validity_bitmap_mda),
error::BITPACK_VALIDITY_BITMAP_ERROR_ID);

auto array_data = arrow::ArrayData::Make(data_type, array_length, {packed_validity_bitmap, data_buffer});
auto time32_array = std::static_pointer_cast<Time32Array>(arrow::MakeArray(array_data));
return std::make_shared<Time32ArrayProxy>(std::move(time32_array));
return make_time_array<arrow::Time32Type>(constructor_arguments);
}
}
28 changes: 28 additions & 0 deletions matlab/src/cpp/arrow/matlab/array/proxy/time64_array.cc
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
// Licensed to the Apache Software Foundation (ASF) under one
// or more contributor license agreements. See the NOTICE file
// distributed with this work for additional information
// regarding copyright ownership. The ASF licenses this file
// to you under the Apache License, Version 2.0 (the
// "License"); you may not use this file except in compliance
// with the License. You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing,
// software distributed under the License is distributed on an
// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
// KIND, either express or implied. See the License for the
// specific language governing permissions and limitations
// under the License.

#include "arrow/matlab/array/proxy/time64_array.h"
#include "arrow/matlab/array/proxy/time_array.h"

namespace arrow::matlab::array::proxy {

// Specialization of NumericArray::Make for arrow::Time64Type
template <>
libmexclass::proxy::MakeResult Time64Array::make(const libmexclass::proxy::FunctionArguments& constructor_arguments) {
return make_time_array<arrow::Time64Type>(constructor_arguments);
}
}
31 changes: 31 additions & 0 deletions matlab/src/cpp/arrow/matlab/array/proxy/time64_array.h
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
// Licensed to the Apache Software Foundation (ASF) under one
// or more contributor license agreements. See the NOTICE file
// distributed with this work for additional information
// regarding copyright ownership. The ASF licenses this file
// to you under the Apache License, Version 2.0 (the
// "License"); you may not use this file except in compliance
// with the License. You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing,
// software distributed under the License is distributed on an
// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
// KIND, either express or implied. See the License for the
// specific language governing permissions and limitations
// under the License.

#pragma once

#include "arrow/matlab/array/proxy/numeric_array.h"

#include "arrow/matlab/api/visibility.h"

namespace arrow::matlab::array::proxy {

using Time64Array = NumericArray<arrow::Time64Type>;

// Specialization of NumericArray::Make for arrow::Time64Type
template<>
ARROW_MATLAB_EXPORT libmexclass::proxy::MakeResult Time64Array::make(const libmexclass::proxy::FunctionArguments& constructor_arguments);
}
72 changes: 72 additions & 0 deletions matlab/src/cpp/arrow/matlab/array/proxy/time_array.h
Original file line number Diff line number Diff line change
@@ -0,0 +1,72 @@
// Licensed to the Apache Software Foundation (ASF) under one
// or more contributor license agreements. See the NOTICE file
// distributed with this work for additional information
// regarding copyright ownership. The ASF licenses this file
// to you under the Apache License, Version 2.0 (the
// "License"); you may not use this file except in compliance
// with the License. You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing,
// software distributed under the License is distributed on an
// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
// KIND, either express or implied. See the License for the
// specific language governing permissions and limitations
// under the License.

#include "arrow/matlab/array/proxy/numeric_array.h"

#include "arrow/matlab/type/time_unit.h"
#include "arrow/util/utf8.h"
#include "arrow/type_traits.h"

namespace arrow::matlab::array::proxy {

template <typename ArrowType>
using is_time = arrow::is_time_type<ArrowType>;

template <typename ArrowType>
using enable_if_time = std::enable_if_t<is_time<ArrowType>::value, bool>;

template <typename ArrowType, enable_if_time<ArrowType> = true>
libmexclass::proxy::MakeResult make_time_array(const libmexclass::proxy::FunctionArguments& constructor_arguments) {
namespace mda = ::matlab::data;
using namespace arrow::matlab::type;
using MatlabBuffer = arrow::matlab::buffer::MatlabBuffer;
using TimeArray = arrow::NumericArray<ArrowType>;
using TimeArrayProxy = proxy::NumericArray<ArrowType>;
using CType = typename arrow::TypeTraits<ArrowType>::CType;

mda::StructArray opts = constructor_arguments[0];

const mda::TypedArray<CType> time_mda = opts[0]["MatlabArray"];
const mda::TypedArray<bool> validity_bitmap_mda = opts[0]["Valid"];
const mda::TypedArray<mda::MATLABString> time_unit_mda = opts[0]["TimeUnit"];

// extract the time unit
const std::u16string& time_unit_utf16 = time_unit_mda[0];
MATLAB_ASSIGN_OR_ERROR(const auto time_unit,
timeUnitFromString(time_unit_utf16),
error::UKNOWN_TIME_UNIT_ERROR_ID);

MATLAB_ERROR_IF_NOT_OK(validateTimeUnit<ArrowType>(time_unit),
error::INVALID_TIME_UNIT);

// create the ArrowType
const auto data_type = std::make_shared<ArrowType>(time_unit);

auto array_length = static_cast<size_t>(time_mda.getNumberOfElements());
auto data_buffer = std::make_shared<MatlabBuffer>(time_mda);

// Pack the validity bitmap values.
MATLAB_ASSIGN_OR_ERROR(auto packed_validity_bitmap,
bit::packValid(validity_bitmap_mda),
error::BITPACK_VALIDITY_BITMAP_ERROR_ID);

auto array_data = arrow::ArrayData::Make(data_type, array_length, {packed_validity_bitmap, data_buffer});
auto time_array = std::static_pointer_cast<TimeArray>(arrow::MakeArray(array_data));
return std::make_shared<TimeArrayProxy>(std::move(time_array));

}
}
4 changes: 3 additions & 1 deletion matlab/src/cpp/arrow/matlab/array/proxy/wrap.cc
Original file line number Diff line number Diff line change
Expand Up @@ -53,10 +53,12 @@ namespace arrow::matlab::array::proxy {
return std::make_shared<proxy::NumericArray<arrow::TimestampType>>(std::static_pointer_cast<arrow::TimestampArray>(array));
case ID::TIME32:
return std::make_shared<proxy::NumericArray<arrow::Time32Type>>(std::static_pointer_cast<arrow::Time32Array>(array));
case ID::TIME64:
return std::make_shared<proxy::NumericArray<arrow::Time64Type>>(std::static_pointer_cast<arrow::Time64Array>(array));
case ID::STRING:
return std::make_shared<proxy::StringArray>(std::static_pointer_cast<arrow::StringArray>(array));
default:
return arrow::Status::NotImplemented("Unsupported DataType: " + array->type()->ToString());
}
}
}
}
1 change: 1 addition & 0 deletions matlab/src/cpp/arrow/matlab/error/error.h
Original file line number Diff line number Diff line change
Expand Up @@ -171,6 +171,7 @@ namespace arrow::matlab::error {
static const char* STRING_BUILDER_APPEND_FAILED = "arrow:matlab:array:string:StringBuilderAppendFailed";
static const char* STRING_BUILDER_FINISH_FAILED = "arrow:matlab:array:string:StringBuilderFinishFailed";
static const char* UKNOWN_TIME_UNIT_ERROR_ID = "arrow:matlab:UnknownTimeUnit";
static const char* INVALID_TIME_UNIT = "arrow:type:InvalidTimeUnit";
static const char* FIELD_FAILED_TO_CREATE_TYPE_PROXY = "arrow:field:FailedToCreateTypeProxy";
static const char* ARRAY_FAILED_TO_CREATE_TYPE_PROXY = "arrow:array:FailedToCreateTypeProxy";
static const char* ARROW_TABULAR_SCHEMA_INVALID_NUMERIC_FIELD_INDEX = "arrow:tabular:schema:InvalidNumericFieldIndex";
Expand Down
2 changes: 2 additions & 0 deletions matlab/src/cpp/arrow/matlab/proxy/factory.cc
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@
#include "arrow/matlab/array/proxy/string_array.h"
#include "arrow/matlab/array/proxy/timestamp_array.h"
#include "arrow/matlab/array/proxy/time32_array.h"
#include "arrow/matlab/array/proxy/time64_array.h"
#include "arrow/matlab/tabular/proxy/record_batch.h"
#include "arrow/matlab/tabular/proxy/schema.h"
#include "arrow/matlab/error/error.h"
Expand Down Expand Up @@ -52,6 +53,7 @@ libmexclass::proxy::MakeResult Factory::make_proxy(const ClassName& class_name,
REGISTER_PROXY(arrow.array.proxy.StringArray , arrow::matlab::array::proxy::StringArray);
REGISTER_PROXY(arrow.array.proxy.TimestampArray, arrow::matlab::array::proxy::NumericArray<arrow::TimestampType>);
REGISTER_PROXY(arrow.array.proxy.Time32Array , arrow::matlab::array::proxy::NumericArray<arrow::Time32Type>);
REGISTER_PROXY(arrow.array.proxy.Time64Array , arrow::matlab::array::proxy::NumericArray<arrow::Time64Type>);
REGISTER_PROXY(arrow.tabular.proxy.RecordBatch , arrow::matlab::tabular::proxy::RecordBatch);
REGISTER_PROXY(arrow.tabular.proxy.Schema , arrow::matlab::tabular::proxy::Schema);
REGISTER_PROXY(arrow.type.proxy.Field , arrow::matlab::type::proxy::Field);
Expand Down
21 changes: 1 addition & 20 deletions matlab/src/cpp/arrow/matlab/type/proxy/time32_type.cc
Original file line number Diff line number Diff line change
Expand Up @@ -16,31 +16,12 @@
// under the License.

#include "arrow/matlab/type/proxy/time32_type.h"
#include "arrow/matlab/type/time_unit.h"
#include "arrow/matlab/error/error.h"
#include "arrow/util/utf8.h"

namespace arrow::matlab::type::proxy {

Time32Type::Time32Type(std::shared_ptr<arrow::Time32Type> time32_type) : TimeType(std::move(time32_type)) {}

libmexclass::proxy::MakeResult Time32Type::make(const libmexclass::proxy::FunctionArguments& constructor_arguments) {
namespace mda = ::matlab::data;

using Time32TypeProxy = arrow::matlab::type::proxy::Time32Type;

mda::StructArray opts = constructor_arguments[0];

const mda::StringArray timeunit_mda = opts[0]["TimeUnit"];

// extract the time unit
const std::u16string& utf16_timeunit = timeunit_mda[0];
MATLAB_ASSIGN_OR_ERROR(const auto timeunit,
arrow::matlab::type::timeUnitFromString(utf16_timeunit),
error::UKNOWN_TIME_UNIT_ERROR_ID);

auto type = arrow::time32(timeunit);
auto time_type = std::static_pointer_cast<arrow::Time32Type>(type);
return std::make_shared<Time32TypeProxy>(std::move(time_type));
return make_time_type<arrow::Time32Type>(constructor_arguments);
}
}
21 changes: 1 addition & 20 deletions matlab/src/cpp/arrow/matlab/type/proxy/time64_type.cc
Original file line number Diff line number Diff line change
Expand Up @@ -16,31 +16,12 @@
// under the License.

#include "arrow/matlab/type/proxy/time64_type.h"
#include "arrow/matlab/type/time_unit.h"
#include "arrow/matlab/error/error.h"
#include "arrow/util/utf8.h"

namespace arrow::matlab::type::proxy {

Time64Type::Time64Type(std::shared_ptr<arrow::Time64Type> time64_type) : TimeType(std::move(time64_type)) {}

libmexclass::proxy::MakeResult Time64Type::make(const libmexclass::proxy::FunctionArguments& constructor_arguments) {
namespace mda = ::matlab::data;

using Time64TypeProxy = arrow::matlab::type::proxy::Time64Type;

mda::StructArray opts = constructor_arguments[0];

const mda::StringArray timeunit_mda = opts[0]["TimeUnit"];

// extract the time unit
const std::u16string& utf16_timeunit = timeunit_mda[0];
MATLAB_ASSIGN_OR_ERROR(const auto timeunit,
arrow::matlab::type::timeUnitFromString(utf16_timeunit),
error::UKNOWN_TIME_UNIT_ERROR_ID);

auto type = arrow::time64(timeunit);
auto time_type = std::static_pointer_cast<arrow::Time64Type>(type);
return std::make_shared<Time64TypeProxy>(std::move(time_type));
return make_time_type<arrow::Time64Type>(constructor_arguments);
}
}
41 changes: 39 additions & 2 deletions matlab/src/cpp/arrow/matlab/type/proxy/time_type.cc
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,10 @@
// under the License.

#include "arrow/matlab/type/proxy/time_type.h"
#include "arrow/matlab/type/proxy/traits.h"
#include "arrow/matlab/type/time_unit.h"
#include "arrow/matlab/error/error.h"
#include "arrow/util/utf8.h"

namespace arrow::matlab::type::proxy {

Expand All @@ -28,10 +32,43 @@ namespace arrow::matlab::type::proxy {
mda::ArrayFactory factory;

auto time_type = std::static_pointer_cast<arrow::TimeType>(data_type);
const auto timeunit = time_type->unit();
const auto time_unit = time_type->unit();
// Cast to uint8_t since there are only four supported TimeUnit enumeration values:
// Nanosecond, Microsecond, Millisecond, Second
auto timeunit_mda = factory.createScalar(static_cast<uint8_t>(timeunit));
auto timeunit_mda = factory.createScalar(static_cast<uint8_t>(time_unit));
context.outputs[0] = timeunit_mda;
}

template <typename ArrowType>
libmexclass::proxy::MakeResult make_time_type(const libmexclass::proxy::FunctionArguments& constructor_arguments) {
namespace mda = ::matlab::data;
using namespace arrow::matlab::type;
using TimeTypeProxy = typename proxy::Traits<ArrowType>::TypeProxy;

mda::StructArray opts = constructor_arguments[0];

const mda::StringArray time_unit_mda = opts[0]["TimeUnit"];

// extract the time unit
const std::u16string& time_unit_utf16 = time_unit_mda[0];
MATLAB_ASSIGN_OR_ERROR(const auto timeunit,
timeUnitFromString(time_unit_utf16),
error::UKNOWN_TIME_UNIT_ERROR_ID);

// validate timeunit
MATLAB_ERROR_IF_NOT_OK(validateTimeUnit<ArrowType>(timeunit),
error::INVALID_TIME_UNIT);

auto type = std::make_shared<ArrowType>(timeunit);
auto time_type = std::static_pointer_cast<ArrowType>(type);
return std::make_shared<TimeTypeProxy>(std::move(time_type));
}

// Trigger code generation for the allowed template specializations using explicit instantiation.
template
libmexclass::proxy::MakeResult make_time_type<arrow::Time32Type>(const libmexclass::proxy::FunctionArguments& constructor_arguments);

template
libmexclass::proxy::MakeResult make_time_type<arrow::Time64Type>(const libmexclass::proxy::FunctionArguments& constructor_arguments);

}
4 changes: 4 additions & 0 deletions matlab/src/cpp/arrow/matlab/type/proxy/time_type.h
Original file line number Diff line number Diff line change
Expand Up @@ -32,4 +32,8 @@ class TimeType : public arrow::matlab::type::proxy::FixedWidthType {
void getTimeUnit(libmexclass::proxy::method::Context& context);
};

template <typename ArrowType>
libmexclass::proxy::MakeResult make_time_type(const libmexclass::proxy::FunctionArguments& constructor_arguments);


}
12 changes: 12 additions & 0 deletions matlab/src/cpp/arrow/matlab/type/proxy/traits.h
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,8 @@

#include "arrow/matlab/type/proxy/primitive_ctype.h"
#include "arrow/matlab/type/proxy/timestamp_type.h"
#include "arrow/matlab/type/proxy/time32_type.h"
#include "arrow/matlab/type/proxy/time64_type.h"
#include "arrow/matlab/type/proxy/string_type.h"

namespace arrow::matlab::type::proxy {
Expand Down Expand Up @@ -87,4 +89,14 @@ namespace arrow::matlab::type::proxy {
struct Traits<arrow::TimestampType> {
using TypeProxy = TimestampType;
};

template <>
struct Traits<arrow::Time32Type> {
using TypeProxy = Time32Type;
};

template <>
struct Traits<arrow::Time64Type> {
using TypeProxy = Time64Type;
};
}
Loading

0 comments on commit faccbef

Please sign in to comment.