Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Parsing local and cloud SEG-Y files with new I/O library (#381)
* Refactor text_header setter type check Removed the list type check from the text_header setter in accessor.py. The application now expects a string input instead of a list, simplifying the validation process. * Refactor text_header setter type check Removed the list type check from the text_header setter in accessor.py. The application now expects a string input instead of a list, simplifying the validation process. * Refactor text_header setter type check Removed the list type check from the text_header setter in accessor.py. The application now expects a string input instead of a list, simplifying the validation process. * Refactor workers for SEG-Y parsing Simplify `header_scan_worker` and `trace_worker` in SEG-Y module by removing unused imports and streamlining parameter list. Update functions to work directly with `SegyFile` instances and clean up data handling logic for efficiency. * Refactor SEG-Y parser and streamline imports Refactor the parsing functions in `src/mdio/segy/parsers.py` to simplify the codebase and improve maintainability. Redundant functions such as `parse_binary_header`, `parse_text_header`, and `get_trace_count` have been removed, while imports have been condensed to only essential modules. The `NUM_CORES` logic is updated to count logical cores instead of just physical ones. * Refactor SEG-Y converter and simplify imports Removed unused imports and functions in the SEG-Y converter module to enhance code maintainability. Simplified the arguments for the `segy_to_mdio` function to increase ease of use and readability. Reduced complexity by utilizing `SegyFile` class for SEG-Y file operations. * Refactor get_grid_plan and remove unused imports The get_grid_plan function in utilities.py has been refactored to accept a SegyFile instance instead of individual parameters for the file path. Unused imports were eliminated, and type checking imports are now conditional, improving readability and modularity. * use NDArray typing since we now return struct * Refactor to use 'segy' instead of 'segyio'. The changes involve major refactoring of the code base to use the 'segy' library instead of 'segyio'. Most notably, this included updating the handling of SEG-Y dtypes, byte order, and trace headers. Unused imports have been removed to clean up the code. A new multiprocessing chunk size has been introduced and set attributes to SegyFile instance instead of passing them as function arguments. * refactor override tests to use ndarray headers instead of a dictionary to make it work with 'segy'. * Remove unit tests for IBM/IEEE conversions and text headers * Refactor and simplify 6D tests related to SEG-Y * Refactor and simplify 6D tests related to SEG-Y * Upgrade segy package version The segy package version has been updated from 0.0.13 to 0.0.14 in the pyproject.toml file. This upgrade was performed to update software dependencies and to integrate the latest bug fixes and features delivered with the new version. * Refactored segy factory creation in mdio_to_segy function A new helper function, 'make_segy_factory', has been created to handle the generation of SegyFactory. This function accepts more parameters to provide better control over the creation of the SEG-Y based on the MDIO metadata. Changes also include updates in import declarations and reorganization of some code blocks in the 'mdio_spec_to_segy' function. * Update segy library version * Multiply sample_interval by 1000 in SegyFactory In the SegyFactory initialization within creation.py, the sample_interval parameter has been modified to be multiplied by 1000. This change ensures that the value is correctly represented in microseconds, aligning with the expected data format. * fix docstring errors * Update dependency package versions * update field name for segy data * import Endianness from new location * use bleeding edge segy during dev * allow configuring endianness on export * update binary header * Update the 'segy' git repository link * Update virtualenv version in constraints.txt * Update poetry version in workflow constraints * update RtD dependencies * switch myst-nb to stable * fix broken tests * fix broken tests * simplify factory usage and fix tests * add original segyio fields as spec * Add pytest-dependency to project dev dependencies * fix: headers were missed due to early return * streamline mdio segy spec * simplify mock 4d generation * enforce mdio segy spec * update type hints to the correct segy type. * revert api * update type hints * remove endian from segy import because its inferred * remove output format from seg-y export. we only export as its set in "binary header" * update endian kwarg name * revert to old api * enable all tests * Update get_grid_plan * Remove unused byte swapping function from segy creation module. * Remove now unused byte utils module * Add temporary safety check ignore for specific CVE The safety check in noxfile.py has been updated to temporarily ignore a specific Common Vulnerabilities and Exposures (CVE) number because it's not deemed critical. A TODO note is added to remind removal of this exception once the issue is resolved. * fix safety ignore syntax * make temp zarr files module scoped * revert to_segy endian api * simplify changes * Correct variable in default chunk selection * Update segy package version in pyproject.toml * use correct spec for factory * use new endian inference from `segy` * get header dtype from spec instead of reading a header * remove unnecessary cast * remove commented line * Implement dynamic CPU count for header parsing * backward_compat: revert text header to write as list[str] instead of str with newline * generate spec as needed and avoid singleton bugs * bump version * add missing return doc * Add future annotations import for type hints
- Loading branch information
