Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Are High Disk Writes Expected? #264

Open
JohnTravolski opened this issue Jun 27, 2024 · 3 comments
Open

Are High Disk Writes Expected? #264

JohnTravolski opened this issue Jun 27, 2024 · 3 comments

Comments

@JohnTravolski
Copy link

JohnTravolski commented Jun 27, 2024

I have been running Bitcoin Core 27.0 with txindex=1 and ElectrumX 1.16.0 on a Samsung 860 Evo SSD on Xubuntu. This is a dedicated machine doing nothing but running those two programs all day. I've also been using smartmontools to monitor the total disk writes once a day over time, and I've noticed the numbers are shockingly high. Obviously this is a concern since SSDs can only perform so many writes before they die.

Unfortunately, I only have a few data points right now, but all of these were recorded AFTER both the initial block download and after the transaction index was finished being built:

30 total gigabytes written to the SSD from June 24th to June 25th
108 total gigabytes written to the SSD from June 25th to June 26th
158 total gigabytes written to the SSD from June 26th to June 27th

To be clear, this is the difference in total GiB (512 bytes * Total LBAs Written /1024**3) written to the disk from day to day, not actual capacity being used. The amount of disk space being used is about what I would expect (increasing a few hundred megabytes per day).

I'm getting the LBAs written from smartmontools using:
sudo smartctl -a /dev/sda
and pulling the Total_LBAs_Written field, in case anybody else wants to try to monitor this for themselves and compare.

Obviously there's a lot more testing I need to do before I can determine what's causing this for sure but of course my intuition is that it's either related to Bitcoin Core or ElectrumX since the machine is dedicated to running them.

It is also possible I have the LBA size wrong for this drive and I'm inflating the gigabytes written without realizing it, but 512 bytes is standard from what I can tell.

Do these disk write numbers seem reasonable to you, or are these way too high?

@JohnTravolski
Copy link
Author

JohnTravolski commented Jun 28, 2024

I started using iotop --accumulated to measure disk writes and after just 15 minutes I'm seeing this:

Disk Writes          Command
3.36 G                python3 /home/user/Desktop/electrumx/electrumx_server
17.97 M               [jdb2/sda2-8]
6.52 M                systemd-journald
3.84 M                bitcoin-qt -min -chain=main [b-scheduler]
3.75 M                bitcoin-qt -min -chain=main [b-msghand]
3.36 M                tor --defaults-torrc /usr/share/tor/tor-service-defaults-torrc -f /etc/tor/torrc --RunAsDaemon 0

So it seems like it is ElectrumX that is doing all the writes. Do I have something configured wrong or is this expected?

My electrumx configuration is the same as in this issue:
#263
It looks like there is a CACHE_MB variable I can set. My system has sufficient RAM (16GB). Should I try increasing this significantly? Might this reduce disk writes? If not, is there something else I should do to limit the total disk writes so I don't chew through the SSD so fast?

@cculianu
Copy link
Collaborator

Are you using rocksdb or leveldb? Might be some leveldb and/or rocksdb setting that can get tuned? No idea.

@JohnTravolski
Copy link
Author

JohnTravolski commented Jun 29, 2024

Are you using rocksdb or leveldb? Might be some leveldb and/or rocksdb setting that can get tuned? No idea.

It is using whatever the default is, so I am pretty sure it is leveldb. I suppose I can try using rocksdb but I'm guessing it will be very similar. I don't even know where to start when it comes trying to tune the settings of each.

I moved the electrumx db folder to a dedicated SSD (not the OS drive, and not the drive bitcoin core is on) just to confirm it is the db folder casuing the problem, and it definitely is. The OS drive gigabytes written flatlined and the new drive jumped. I started monitoring the writes by the hour on this new drive and it seems to be averaging at about 5 GB per hour. It is pretty continuous for the most part.

image

Is anyone else running electrumx observing this for themselves? I'm wondering if I have something configured wrong and it's only me.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants