Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Test of using BTRFS #101

Open
wants to merge 18 commits into
base: master
Choose a base branch
from
Open

Test of using BTRFS #101

wants to merge 18 commits into from

Conversation

rtgoodwin
Copy link
Contributor

Take a look at this @cimryan please.

Changes:

  • backingfiles is created as btrfs (and tools are installed)
  • vfat disks are mounted at the start of the loop instead of in the loop. My thinking is that the normal pattern of the Pi being shut down and restarted with the car is basically a given. So fix them once each "boot". I don't think this should be needed during the archive mount and copy
  • snapshot is created/mounted/deleted during archiving, files are copied from there, and drives are never removed from the host. To see how stability is improved/isn't.

Problems:

  • Since it's a snapshot, files are copied not moved. If we go down this path, we can figure out how to deal with that. Some people want them copied not moved anyway, so a purge algorithm might be better anyway.
  • I reverted to using your partitioning logic; parted doesn't seem to be loving the way I was doing it, or I ended up creating a bug. I'll figure this out separately as I like the simpler logic
  • I'm missing something basic but campercent less than 100 ends up causing the music disk to be created too big. Again probably some basic thing I missed

It's a proof of concept, but I like it, if it tests stably (and better than using a simultaneous ro mount which is another option to explore). btrfs is used in production pretty heavily so I don't feel too worried about it...yet.

- The BACKINGFILES check will throw an error as -e can't be used with globs.
- switch filesystem to btrfs

Leaving fsck alone for now; shouldn't be needed but also shouldn't hurt.
- Ensure snapshot remove is committed
- check source and target files and sizes before copy
- reserve some extra space on the backingfiles partition for snapshots
This should never be mounted at boot; had it there for testing purposes.

fix_errors_in_cam_file
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When a snapshot is taken of the filesystem holding the cam backingfile there's no guarantee that the contents of the backing file represent a valid file system, is there? I.e. the kernel could be writing metadata into the virtual file system, which could be partially committed to the backing file at the instant the snapshot is captured. It seems that an fsck should be performed on the FS contained within the snap before its contents are accessed.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's fine except the snap is read-only; I've done some simultaneous reading/writing and it seems to hold up. We could definitely add in fsck w/o repair and see how often there's a problem. Thoughts?

setup/pi/setup-teslausb Outdated Show resolved Hide resolved
@cimryan
Copy link
Owner

cimryan commented Nov 24, 2018

Using btrfs as a means of copying the cam files while the drive is still being used is clever.

Moving saved clips off the drive will require disconnecting the virtual drive from the Tesla. It seems reasonable to disconnect the drive just before archiving.

Copying the ephemeral cam files seems like it would be another set of steps in archiveloop.

@rtgoodwin
Copy link
Contributor Author

Moving saved clips off the drive will require disconnecting the virtual drive from the Tesla. It seems reasonable to disconnect the drive just before archiving.

Copying the ephemeral cam files seems like it would be another set of steps in archiveloop.

I have a few thoughts here; for example, we could have pre-mount actions that move off existing files, and/or a configurable prune routine (by space, by date, etc.).

All things being equal, I don't love disconnecting the drive, but I can't empirically prove that it's not the best way to do things. :) It just seems cleaner to say "once the drive is online, it has one goal: stay online and stable", since its purpose is to be as available as possible in case the footage is needed.

@rtgoodwin
Copy link
Contributor Author

Updated commit to remove the flagged items.

We can wait to merge this until we decide on how to handle moves/archiving, or to user branch now; it's not terribly unclean as is.

I'd love to hear your thoughts on stability if you get a chance to test it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants