Fix JDC fallback to solo-mining #1055

lorbax · 2024-07-16T11:35:17Z

If the JDC switches to solo_mining, then it initializes as solo mining (using initialize_jd_as_solo_miner).
This function initialize a DownstreamMiningNode with jd field set to None. This is the right behaviour, but makes panic this unwrap.
Since we are doing solo mining, it should be care of the downstream to change the coinbase downstream, instead of wrapping a None, the coinbase_tx_prefix and suffix are ignored.

Perhaps this is not the best way to do that, feedback needed.

In the commit message is writtend how to test this PR.

This PR is on top of PR1001.
This PR closes issue 844

How to test this PR:
The testing environment is very similar to the one described on PR1001. The only difference is that here we do not have a "good" pool to fallback into. So apply suggestions on PR1001 with the only difference that the jdc config is

[[upstreams]]
authority_pubkey = "9auqWEzQDVyd2oe1JVGFLMLHZtCo2FFqZwtKA5gd9xbuEu7PH72"
pool_address = "127.0.0.1:44254"
jd_address = "127.0.0.1:34264"
pool_signature = "Stratum v2 SRI Pool"

(you can avoid running a "good" pool then)

EDIT: the MG test is removed

github-actions · 2024-07-16T11:39:22Z

Bencher

Report	Fri, August 16, 2024 at 22:19:57 UTC
Project	Stratum v2 (SRI)
Branch	jdc-gets-stuck-in-fallback-to-solo-mining
Testbed	sv1

Click to view all benchmark results

Benchmark	Estimated Cycles	Estimated Cycles Results estimated cycles \| (Δ%)	Estimated Cycles Upper Boundary estimated cycles \| (%)	Instructions	Instructions Results instructions \| (Δ%)	Instructions Upper Boundary instructions \| (%)	L1 Accesses	L1 Accesses Results accesses \| (Δ%)	L1 Accesses Upper Boundary accesses \| (%)	L2 Accesses	L2 Accesses Results accesses \| (Δ%)	L2 Accesses Upper Boundary accesses \| (%)	RAM Accesses	RAM Accesses Results accesses \| (Δ%)	RAM Accesses Upper Boundary accesses \| (%)
get_authorize	✅ (view plot)	8,542.00 (+1.05%)	8,725.16 (97.90%)	✅ (view plot)	3,746.00 (+0.08%)	3,848.15 (97.35%)	✅ (view plot)	5,247.00 (-0.02%)	5,391.98 (97.31%)	✅ (view plot)	8.00 (+1.13%)	10.16 (78.72%)	✅ (view plot)	93.00 (+2.84%)	94.52 (98.40%)
get_submit	✅ (view plot)	95,507.00 (-0.05%)	96,092.22 (99.39%)	✅ (view plot)	59,439.00 (-0.05%)	59,750.85 (99.48%)	✅ (view plot)	85,352.00 (-0.06%)	85,800.13 (99.48%)	✅ (view plot)	57.00 (+4.77%)	62.98 (90.51%)	✅ (view plot)	282.00 (-0.08%)	287.45 (98.10%)
get_subscribe	✅ (view plot)	7,991.00 (+0.11%)	8,250.18 (96.86%)	✅ (view plot)	2,841.00 (+0.31%)	2,932.36 (96.88%)	✅ (view plot)	3,971.00 (+0.34%)	4,090.60 (97.08%)	✅ (view plot)	13.00 (-16.89%)	19.81 (65.62%)	✅ (view plot)	113.00 (+0.23%)	116.73 (96.80%)
serialize_authorize	✅ (view plot)	12,261.00 (+0.31%)	12,506.09 (98.04%)	✅ (view plot)	5,317.00 (+0.06%)	5,419.15 (98.11%)	✅ (view plot)	7,411.00 (+0.00%)	7,555.42 (98.09%)	✅ (view plot)	11.00 (+3.66%)	13.34 (82.44%)	✅ (view plot)	137.00 (+0.76%)	140.70 (97.37%)
serialize_deserialize_authorize	✅ (view plot)	24,562.00 (+0.30%)	24,720.55 (99.36%)	✅ (view plot)	9,898.00 (-0.05%)	10,019.71 (98.79%)	✅ (view plot)	13,957.00 (-0.10%)	14,142.70 (98.69%)	✅ (view plot)	35.00 (-3.22%)	41.14 (85.09%)	✅ (view plot)	298.00 (+0.90%)	298.92 (99.69%)
serialize_deserialize_handle_authorize	✅ (view plot)	30,321.00 (+0.49%)	30,387.81 (99.78%)	✅ (view plot)	12,101.00 (+0.03%)	12,203.15 (99.16%)	✅ (view plot)	17,111.00 (-0.05%)	17,268.38 (99.09%)	✅ (view plot)	66.00 (+10.49%)	66.86 (98.72%)	✅ (view plot)	368.00 (+0.99%)	368.69 (99.81%)
serialize_deserialize_handle_submit	✅ (view plot)	126,324.00 (-0.07%)	126,991.57 (99.47%)	✅ (view plot)	73,224.00 (-0.04%)	73,587.22 (99.51%)	✅ (view plot)	104,944.00 (-0.04%)	105,466.97 (99.50%)	✅ (view plot)	125.00 (+3.78%)	130.97 (95.44%)	✅ (view plot)	593.00 (-0.34%)	599.64 (98.89%)
serialize_deserialize_handle_subscribe	✅ (view plot)	27,503.00 (+0.13%)	27,609.65 (99.61%)	✅ (view plot)	9,643.00 (+0.09%)	9,734.36 (99.06%)	✅ (view plot)	13,638.00 (+0.09%)	13,765.17 (99.08%)	✅ (view plot)	64.00 (-2.50%)	73.08 (87.57%)	✅ (view plot)	387.00 (+0.25%)	388.88 (99.52%)
serialize_deserialize_submit	✅ (view plot)	114,955.00 (-0.11%)	115,623.52 (99.42%)	✅ (view plot)	68,001.00 (-0.08%)	68,377.56 (99.45%)	✅ (view plot)	97,555.00 (-0.10%)	98,119.10 (99.43%)	✅ (view plot)	71.00 (+3.08%)	75.66 (93.84%)	✅ (view plot)	487.00 (-0.25%)	492.43 (98.90%)
serialize_deserialize_subscribe	✅ (view plot)	22,884.00 (+0.00%)	23,107.05 (99.03%)	✅ (view plot)	8,195.00 (+0.08%)	8,290.13 (98.85%)	✅ (view plot)	11,544.00 (+0.09%)	11,673.51 (98.89%)	✅ (view plot)	35.00 (-9.67%)	44.02 (79.51%)	✅ (view plot)	319.00 (+0.07%)	321.86 (99.11%)
serialize_submit	✅ (view plot)	99,786.00 (-0.11%)	100,430.20 (99.36%)	✅ (view plot)	61,483.00 (-0.05%)	61,799.66 (99.49%)	✅ (view plot)	88,201.00 (-0.06%)	88,652.95 (99.49%)	✅ (view plot)	56.00 (+2.37%)	63.43 (88.29%)	✅ (view plot)	323.00 (-0.56%)	329.69 (97.97%)
serialize_subscribe	✅ (view plot)	11,294.00 (-0.34%)	11,600.10 (97.36%)	✅ (view plot)	4,188.00 (+0.21%)	4,279.36 (97.87%)	✅ (view plot)	5,829.00 (+0.23%)	5,949.28 (97.98%)	✅ (view plot)	15.00 (-6.77%)	18.84 (79.62%)	✅ (view plot)	154.00 (-0.86%)	160.04 (96.23%)

Bencher - Continuous Benchmarking
View Public Perf Page
Docs | Repo | Chat | Help

github-actions · 2024-07-16T11:39:22Z

Bencher

Report	Fri, August 16, 2024 at 22:20:02 UTC
Project	Stratum v2 (SRI)
Branch	1055/merge
Testbed	sv1

Click to view all benchmark results

Benchmark	Latency	Latency Results nanoseconds (ns) \| (Δ%)	Latency Upper Boundary nanoseconds (ns) \| (%)
client-submit-serialize	✅ (view plot)	6,545.70 (-3.95%)	7,368.15 (88.84%)
client-submit-serialize-deserialize	✅ (view plot)	7,370.20 (-4.72%)	8,325.55 (88.53%)
client-submit-serialize-deserialize-handle/client-submit-serialize-deserialize-handle	✅ (view plot)	8,021.30 (-3.55%)	8,849.29 (90.64%)
client-sv1-authorize-serialize-deserialize-handle/client-sv1-authorize-serialize-deserialize-handle	✅ (view plot)	888.59 (-1.16%)	929.87 (95.56%)
client-sv1-authorize-serialize-deserialize/client-sv1-authorize-serialize-deserialize	✅ (view plot)	699.99 (-0.06%)	727.81 (96.18%)
client-sv1-authorize-serialize/client-sv1-authorize-serialize	✅ (view plot)	245.49 (-1.33%)	260.62 (94.20%)
client-sv1-get-authorize/client-sv1-get-authorize	✅ (view plot)	156.18 (-0.97%)	166.10 (94.03%)
client-sv1-get-submit	✅ (view plot)	6,291.80 (-4.47%)	7,138.01 (88.14%)
client-sv1-get-subscribe/client-sv1-get-subscribe	✅ (view plot)	274.16 (-1.73%)	292.09 (93.86%)
client-sv1-subscribe-serialize-deserialize-handle/client-sv1-subscribe-serialize-deserialize-handle	✅ (view plot)	748.04 (-0.48%)	785.06 (95.28%)
client-sv1-subscribe-serialize-deserialize/client-sv1-subscribe-serialize-deserialize	✅ (view plot)	622.94 (+0.69%)	654.38 (95.20%)
client-sv1-subscribe-serialize/client-sv1-subscribe-serialize	✅ (view plot)	204.37 (-0.80%)	218.05 (93.73%)

Bencher - Continuous Benchmarking
View Public Perf Page
Docs | Repo | Chat | Help

GitGab19 · 2024-07-17T14:51:19Z

Since this PR is built on PR #1001, it's also blocked by #1059

Shourya742

utack. Few nits. Can we remove these lock files also.

Shourya742 · 2024-08-04T18:10:47Z

roles/jd-client/src/lib/downstream.rs

+                    jd.safe_lock(|jd| jd.coinbase_tx_prefix = job.coinbase_tx_prefix.clone())
+                        .unwrap();
+                    jd.safe_lock(|jd| jd.coinbase_tx_suffix = job.coinbase_tx_suffix.clone())
+                        .unwrap();


Suggested change

jd.safe_lock(|jd| jd.coinbase_tx_prefix = job.coinbase_tx_prefix.clone())

.unwrap();

jd.safe_lock(|jd| jd.coinbase_tx_suffix = job.coinbase_tx_suffix.clone())

.unwrap();

jd.safe_lock(|jd| {

jd.coinbase_tx_prefix = job.coinbase_tx_prefix.clone();

jd.coinbase_tx_suffix = job.coinbase_tx_suffix.clone();

}).unwrap();

I don't commit this suggestion because of the policies on how nice commit history must be, if you need credits for this mod I can add it in the commit message. Let me know.

No credit needed. I see you included the suggestion in the latest push, and it works perfectly.

github-actions · 2024-08-05T08:53:38Z

Bencher

Report	Fri, August 16, 2024 at 22:19:53 UTC
Project	Stratum v2 (SRI)
Branch	jdc-gets-stuck-in-fallback-to-solo-mining
Testbed	sv2

Click to view all benchmark results

Benchmark	Latency	Latency Results nanoseconds (ns) \| (Δ%)	Latency Upper Boundary nanoseconds (ns) \| (%)
client_sv2_handle_message_common	✅ (view plot)	44.85 (+0.60%)	45.24 (99.14%)
client_sv2_handle_message_mining	✅ (view plot)	74.31 (+1.77%)	79.92 (92.98%)
client_sv2_mining_message_submit_standard	✅ (view plot)	14.63 (-0.11%)	14.69 (99.63%)
client_sv2_mining_message_submit_standard_serialize	✅ (view plot)	267.44 (+0.91%)	284.00 (94.17%)
client_sv2_mining_message_submit_standard_serialize_deserialize	✅ (view plot)	592.73 (-0.23%)	625.30 (94.79%)
client_sv2_open_channel	✅ (view plot)	165.41 (-0.11%)	172.86 (95.69%)
client_sv2_open_channel_serialize	✅ (view plot)	271.23 (-3.55%)	294.05 (92.24%)
client_sv2_open_channel_serialize_deserialize	✅ (view plot)	376.50 (-0.26%)	419.17 (89.82%)
client_sv2_setup_connection	✅ (view plot)	159.78 (-2.44%)	174.64 (91.49%)
client_sv2_setup_connection_serialize	✅ (view plot)	448.21 (-4.86%)	506.95 (88.41%)
client_sv2_setup_connection_serialize_deserialize	✅ (view plot)	1,002.40 (+2.98%)	1,042.40 (96.16%)

Bencher - Continuous Benchmarking
View Public Perf Page
Docs | Repo | Chat | Help

github-actions · 2024-08-05T08:53:39Z

Bencher

Report	Fri, August 16, 2024 at 22:20:00 UTC
Project	Stratum v2 (SRI)
Branch	jdc-gets-stuck-in-fallback-to-solo-mining
Testbed	sv2

Click to view all benchmark results

Benchmark	Estimated Cycles	Estimated Cycles Results estimated cycles \| (Δ%)	Estimated Cycles Upper Boundary estimated cycles \| (%)	Instructions	Instructions Results instructions \| (Δ%)	Instructions Upper Boundary instructions \| (%)	L1 Accesses	L1 Accesses Results accesses \| (Δ%)	L1 Accesses Upper Boundary accesses \| (%)	L2 Accesses	L2 Accesses Results accesses \| (Δ%)	L2 Accesses Upper Boundary accesses \| (%)	RAM Accesses	RAM Accesses Results accesses \| (Δ%)	RAM Accesses Upper Boundary accesses \| (%)
client_sv2_handle_message_common	✅ (view plot)	2,141.00 (+3.62%)	2,172.89 (98.53%)	✅ (view plot)	473.00 (+0.40%)	485.45 (97.44%)	✅ (view plot)	731.00 (-0.08%)	752.79 (97.10%)	✅ (view plot)	9.00 (+16.53%)	12.84 (70.10%)	✅ (view plot)	39.00 (+5.32%)	39.58 (98.54%)
client_sv2_handle_message_mining	✅ (view plot)	8,223.00 (+0.25%)	8,331.28 (98.70%)	✅ (view plot)	2,143.00 (+0.61%)	2,172.01 (98.66%)	✅ (view plot)	3,173.00 (+0.76%)	3,217.29 (98.62%)	✅ (view plot)	30.00 (-20.02%)	44.31 (67.71%)	✅ (view plot)	140.00 (+0.71%)	141.97 (98.61%)
client_sv2_mining_message_submit_standard	✅ (view plot)	6,350.00 (+1.02%)	6,401.26 (99.20%)	✅ (view plot)	1,756.00 (+0.30%)	1,764.19 (99.54%)	✅ (view plot)	2,560.00 (+0.21%)	2,575.00 (99.42%)	✅ (view plot)	16.00 (-10.03%)	23.06 (69.40%)	✅ (view plot)	106.00 (+1.85%)	107.14 (98.94%)
client_sv2_mining_message_submit_standard_serialize	✅ (view plot)	14,869.00 (+0.54%)	15,031.98 (98.92%)	✅ (view plot)	4,700.00 (+0.11%)	4,708.19 (99.83%)	✅ (view plot)	6,759.00 (+0.06%)	6,774.26 (99.77%)	✅ (view plot)	47.00 (-1.26%)	52.56 (89.42%)	✅ (view plot)	225.00 (+1.02%)	229.78 (97.92%)
client_sv2_mining_message_submit_standard_serialize_deserialize	✅ (view plot)	27,643.00 (+0.50%)	27,858.82 (99.23%)	✅ (view plot)	10,591.00 (+0.38%)	10,595.25 (99.96%)	✅ (view plot)	15,408.00 (+0.36%)	15,412.01 (99.97%)	✅ (view plot)	81.00 (-3.18%)	89.01 (91.00%)	✅ (view plot)	338.00 (+0.82%)	344.99 (97.97%)
client_sv2_open_channel	✅ (view plot)	4,341.00 (-2.70%)	4,635.53 (93.65%)	✅ (view plot)	1,461.00 (+0.05%)	1,473.29 (99.17%)	✅ (view plot)	2,161.00 (+0.32%)	2,173.51 (99.42%)	✅ (view plot)	9.00 (-22.92%)	15.15 (59.41%)	✅ (view plot)	61.00 (-5.08%)	69.02 (88.38%)
client_sv2_open_channel_serialize	✅ (view plot)	14,096.00 (-0.65%)	14,446.18 (97.58%)	✅ (view plot)	5,064.00 (+0.01%)	5,076.29 (99.76%)	✅ (view plot)	7,321.00 (+0.03%)	7,339.16 (99.75%)	✅ (view plot)	39.00 (+4.84%)	41.68 (93.58%)	✅ (view plot)	188.00 (-1.54%)	198.46 (94.73%)
client_sv2_open_channel_serialize_deserialize	✅ (view plot)	22,649.00 (+0.03%)	22,991.09 (98.51%)	✅ (view plot)	8,027.00 (+0.44%)	8,032.11 (99.94%)	✅ (view plot)	11,679.00 (+0.46%)	11,681.83 (99.98%)	✅ (view plot)	73.00 (-1.39%)	83.14 (87.80%)	✅ (view plot)	303.00 (-0.38%)	313.82 (96.55%)
client_sv2_setup_connection	✅ (view plot)	4,715.00 (+0.36%)	4,762.04 (99.01%)	✅ (view plot)	1,502.00 (+0.05%)	1,514.29 (99.19%)	✅ (view plot)	2,275.00 (-0.07%)	2,297.39 (99.03%)	✅ (view plot)	12.00 (+23.81%)	14.65 (81.90%)	✅ (view plot)	68.00 (+0.29%)	69.53 (97.80%)
client_sv2_setup_connection_serialize	✅ (view plot)	16,172.00 (-0.43%)	16,473.03 (98.17%)	✅ (view plot)	5,963.00 (+0.01%)	5,975.29 (99.79%)	✅ (view plot)	8,657.00 (+0.02%)	8,676.20 (99.78%)	✅ (view plot)	47.00 (+4.27%)	49.62 (94.73%)	✅ (view plot)	208.00 (-1.10%)	217.08 (95.82%)
client_sv2_setup_connection_serialize_deserialize	✅ (view plot)	35,632.00 (+0.25%)	35,742.45 (99.69%)	✅ (view plot)	14,855.00 (+0.24%)	14,860.08 (99.97%)	✅ (view plot)	21,817.00 (+0.25%)	21,822.23 (99.98%)	✅ (view plot)	96.00 (-3.42%)	111.48 (86.12%)	✅ (view plot)	381.00 (+0.39%)	384.18 (99.17%)

Bencher - Continuous Benchmarking
View Public Perf Page
Docs | Repo | Chat | Help

Shourya742

Utack. A few more nits. The rest looks decent code-wise. I will test the functionality and get back to you soon. Can you delete rest of the lock files.

roles/translator/src/lib/downstream_sv1/downstream.rs

test/config/change-upstream/jdc-config-local-example-change-upstream-solo-fallback.toml

roles/translator/src/lib/proxy/bridge.rs

GitGab19

tACK 👏
I left just one comment, if you could clarify it it would be great, thanks @lorbax

roles/jd-client/src/lib/downstream.rs

fixed the fallback to solo-mining in the case that the upstream sends a `SubmitShareError` on a valid share AND there are no other available upstreams in the JDC config.

fmt

GitGab19 linked an issue Jul 16, 2024 that may be closed by this pull request

JDC is not doing fallback to solo-mining #844

Closed

Shourya742 suggested changes Aug 4, 2024

View reviewed changes

lorbax force-pushed the jdc-gets-stuck-in-fallback-to-solo-mining branch from 54fb3e8 to a3cf8ce Compare August 5, 2024 08:49

lorbax force-pushed the jdc-gets-stuck-in-fallback-to-solo-mining branch 2 times, most recently from f42eec5 to 1993eaf Compare August 5, 2024 09:04

Shourya742 suggested changes Aug 5, 2024

View reviewed changes

roles/translator/src/lib/downstream_sv1/downstream.rs Outdated Show resolved Hide resolved

test/config/change-upstream/jdc-config-local-example-change-upstream-solo-fallback.toml Outdated Show resolved Hide resolved

roles/translator/src/lib/proxy/bridge.rs Outdated Show resolved Hide resolved

lorbax force-pushed the jdc-gets-stuck-in-fallback-to-solo-mining branch 4 times, most recently from 70a17d7 to 341d1e5 Compare August 13, 2024 09:43

plebhash mentioned this pull request Aug 13, 2024

issues lacking CI #1077

Open

lorbax force-pushed the jdc-gets-stuck-in-fallback-to-solo-mining branch from 9067613 to 58c70a4 Compare August 14, 2024 09:23

GitGab19 reviewed Aug 14, 2024

View reviewed changes

roles/jd-client/src/lib/downstream.rs Show resolved Hide resolved

pavlenex mentioned this pull request Aug 14, 2024

test: timeout sv-1 test #1099

Merged

GitGab19 force-pushed the jdc-gets-stuck-in-fallback-to-solo-mining branch from 5f1c25d to 2d0f169 Compare August 16, 2024 22:09

lorbax and others added 2 commits August 17, 2024 00:10

jdc fallback to solo-mining

d479f94

fixed the fallback to solo-mining in the case that the upstream sends a `SubmitShareError` on a valid share AND there are no other available upstreams in the JDC config.

take first coinbase output before encoding - solo mining

cd493ee

fmt

GitGab19 force-pushed the jdc-gets-stuck-in-fallback-to-solo-mining branch from 2d0f169 to cd493ee Compare August 16, 2024 22:10

plebhash approved these changes Aug 17, 2024

View reviewed changes

plebhash merged commit 51c0330 into stratum-mining:dev Aug 17, 2024
33 checks passed

plebhash mentioned this pull request Aug 18, 2024

Release 1.0.2 #1112

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fix JDC fallback to solo-mining #1055

Fix JDC fallback to solo-mining #1055

lorbax commented Jul 16, 2024 •

edited

Loading

github-actions bot commented Jul 16, 2024 •

edited

Loading

github-actions bot commented Jul 16, 2024 •

edited

Loading

GitGab19 commented Jul 17, 2024

Shourya742 left a comment •

edited

Loading

Shourya742 Aug 4, 2024

lorbax Aug 5, 2024

Shourya742 Aug 5, 2024

github-actions bot commented Aug 5, 2024 •

edited

Loading

github-actions bot commented Aug 5, 2024 •

edited

Loading

Shourya742 left a comment •

edited

Loading

GitGab19 left a comment

Fix JDC fallback to solo-mining #1055

Fix JDC fallback to solo-mining #1055

Conversation

lorbax commented Jul 16, 2024 • edited Loading

github-actions bot commented Jul 16, 2024 • edited Loading

Bencher

github-actions bot commented Jul 16, 2024 • edited Loading

Bencher

GitGab19 commented Jul 17, 2024

Shourya742 left a comment • edited Loading

Choose a reason for hiding this comment

Shourya742 Aug 4, 2024

Choose a reason for hiding this comment

lorbax Aug 5, 2024

Choose a reason for hiding this comment

Shourya742 Aug 5, 2024

Choose a reason for hiding this comment

github-actions bot commented Aug 5, 2024 • edited Loading

Bencher

github-actions bot commented Aug 5, 2024 • edited Loading

Bencher

Shourya742 left a comment • edited Loading

Choose a reason for hiding this comment

GitGab19 left a comment

Choose a reason for hiding this comment

lorbax commented Jul 16, 2024 •

edited

Loading

github-actions bot commented Jul 16, 2024 •

edited

Loading

github-actions bot commented Jul 16, 2024 •

edited

Loading

Shourya742 left a comment •

edited

Loading

github-actions bot commented Aug 5, 2024 •

edited

Loading

github-actions bot commented Aug 5, 2024 •

edited

Loading

Shourya742 left a comment •

edited

Loading