[2pt] PR: docker file for OWP, non root user

[RobHanna-NOAA]

For security reasons, we needed to create a docker image that does not use the root user in anyway. The new Dockerfile.prod file is to be used when we want to use a non-root user. The original Dockerfile has been renamed to Dockerfile.dev and will continue to use it's root users which has no problems with interacting with external mounts.

Note: Re: using pip or pipenv installs.
In the Dockerfile.prod, you can not do installs or update using either pipenv or pip. Those types of tests and adjustments need to be done in the Dockerfile.dev. Dockerfile.dev will also allow change to the Pipfile and Pipfile.lock . Both docker files share the Pipfiles so it should be just fine.

File Renames

• Was: Dockerfile, now Dockerfile.dev

Additions

• owp.Dockerfile: as described

Changes

• README.md: change notes from phrase Dockerfile to Dockerfile.dev. Also added some notes about the new convention of outputs no longer starting with fim_ but now hand_
• fim_pipeline.sh: Change for the new Dockerfile.prod for permissions.
• fim_post_processing.sh: Change for the new Dockerfile.prod for permissions.
• fim_pre_processing.sh: Change for the new Dockerfile.prod for permissions.
• fim_process_unit_wb.sh: Change for the new Dockerfile.prod for permissions.

Testing

• Tested on various conditions against owp servers, including:
  • incorrect permissions on mounted folders
  • incorrect group assigned to mounted folders
  • using the internal fim_temp docker container folder when a fim_temp folder is not mounted.
  • ensure files and folders are being cleaned up correctly in outputs_temp and outputs (when applicable)

Deployment Plan (For developer use)

How does the changes affect the product?

• Code only?
• If applicable, has a deployment plan be created with the deployment person/team?
• Require new or adjusted data inputs? Does it have start, end and duration code (in UTC)?
• If new or updated data sets, has the FIM code been updated and tested with the new/adjusted data (subset is fine, but must be a subset of the new data)?
• Require new pre-clip set?
• - sort off. new dockerfile. No new packages. Needs to be deployed to OWP servers only and remove all other current OWP images. EC2's and AWS need not change at this time.

Issuer Checklist (For developer use)

You may update this checklist before and/or after creating the PR. If you're unsure about any of them, please ask, we're here to help! These items are what we are going to look for before merging your code.

• Informative and human-readable title, using the format: [_pt] PR: <description>
• n/a - Links are provided if this PR resolves an issue, or depends on another other PR
• If submitting a PR to the dev branch (the default branch), you have a descriptive Feature Branch name using the format: dev-<description-of-change> (e.g. dev-revise-levee-masking)
• Changes are limited to a single goal (no scope creep)
• The feature branch you're submitting as a PR is up to date (merged) with the latest dev branch
• pre-commit hooks were run locally
• Any change in functionality is tested
• New functions are documented (with a description, list of inputs, and expected output)
• n/a - Placeholder code is flagged / future todos are captured in comments
• CHANGELOG updated with template version number, e.g. 4.x.x.x
• Add yourself as an assignee in the PR as well as the FIM Technical Lead

Merge Checklist (For Technical Lead use only)

• Update CHANGELOG with latest version number and merge date
• Update the Citation.cff file to reflect the latest version number in the CHANGELOG
• If applicable, update README with major alterations

[mluck]

I got an error running fim_pipeline.sh. Merging with dev (v4.5.11.1) should fix this error.

Create list file of branch ids for 03100204
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/dist-packages/pandas/core/indexes/base.py", line 3652, in get_loc
    return self._engine.get_loc(casted_key)
  File "pandas/_libs/index.pyx", line 147, in pandas._libs.index.IndexEngine.get_loc
  File "pandas/_libs/index.pyx", line 176, in pandas._libs.index.IndexEngine.get_loc
  File "pandas/_libs/hashtable_class_helper.pxi", line 7080, in pandas._libs.hashtable.PyObjectHashTable.get_item
  File "pandas/_libs/hashtable_class_helper.pxi", line 7088, in pandas._libs.hashtable.PyObjectHashTable.get_item
KeyError: 'levpa_id'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/foss_fim/src/generate_branch_list.py", line 51, in <module>
    generate_branch_list(**args)
  File "/foss_fim/src/generate_branch_list.py", line 32, in generate_branch_list
    stream_network_dissolved = stream_network_dissolved.loc[:, branch_id_attribute]
  File "/usr/local/lib/python3.10/dist-packages/pandas/core/indexing.py", line 1097, in __getitem__
    return self._getitem_tuple(key)
  File "/usr/local/lib/python3.10/dist-packages/pandas/core/indexing.py", line 1280, in _getitem_tuple
    return self._getitem_lowerdim(tup)
  File "/usr/local/lib/python3.10/dist-packages/pandas/core/indexing.py", line 1000, in _getitem_lowerdim
    section = self._getitem_axis(key, axis=i)
  File "/usr/local/lib/python3.10/dist-packages/pandas/core/indexing.py", line 1343, in _getitem_axis
    return self._get_label(key, axis=axis)
  File "/usr/local/lib/python3.10/dist-packages/pandas/core/indexing.py", line 1293, in _get_label
    return self.obj.xs(label, axis=axis)
  File "/usr/local/lib/python3.10/dist-packages/pandas/core/generic.py", line 4082, in xs
    return self[key]
  File "/usr/local/lib/python3.10/dist-packages/geopandas/geodataframe.py", line 1750, in __getitem__
    result = super().__getitem__(key)
  File "/usr/local/lib/python3.10/dist-packages/pandas/core/frame.py", line 3761, in __getitem__
    indexer = self.columns.get_loc(key)
  File "/usr/local/lib/python3.10/dist-packages/pandas/core/indexes/base.py", line 3654, in get_loc
    raise KeyError(key) from err
KeyError: 'levpa_id'
Command exited with non-zero status 1
	Command being timed: "/foss_fim/src/run_unit_wb.sh"
	User time (seconds): 11.25
	System time (seconds): 5.85
	Percent of CPU this job got: 247%
	Elapsed (wall clock) time (h:mm:ss or m:ss): 0:06.90
	Average shared text size (kbytes): 0
	Average unshared data size (kbytes): 0
	Average stack size (kbytes): 0
	Average total size (kbytes): 0
	Maximum resident set size (kbytes): 272148
	Average resident set size (kbytes): 0
	Major (requiring I/O) page faults: 0
	Minor (reclaiming a frame) page faults: 107405
	Voluntary context switches: 929
	Involuntary context switches: 15738
	Swaps: 0
	File system inputs: 127496
	File system outputs: 133096
	Socket messages sent: 0
	Socket messages received: 0
	Signals delivered: 0
	Page size (bytes): 4096
	Exit status: 1

***** An error has occured  *****

[mluck]

First is current dev container (v4.5.10.0), second is this PR:

[mluck]

This should be changed to Dockerfile.prod.

[mluck]

Should USER be set earlier in the Dockerfile? I think the convention is to set USER as early in the Dockerfile as possible (i.e., after root is used to install necessary packages) to accidentally allow permissions. If USER is set after the ENTRYPOINT is there a chance that the command prompt in the container is not RuntimeUser but root?