From 04c9306443abcf4add70015cf04ed39feb561809 Mon Sep 17 00:00:00 2001 From: sstone Date: Mon, 29 May 2023 16:29:07 +0200 Subject: [PATCH 1/8] Make Eclair manage bitcoin core's wallet private keys We create an empty watch-only wallet and import public descriptors generated by Eclair. Bitcoin Core can fund transaction and manage utxos, but can no longer sign transactions. * Check that spent amounts and utxos are consistent before we sign a PSBT PSBT utxo fields include the amount that are being spent by the PSBT inputs, but there is a "fee attack" where using amounts that are lower than what is actually spent may make us sign a tx that spends much more in fees than we think. * Check that non-segwit uxto has been provided and inputs are signed with SIGHASH_ALL * Verify that Bitcoin Core's fee match what we specified When we call Bitcoin Core's `fundrawtransaction` RPC method, we check that the fee that we pay match the fee rate that we requested. The fee is computed using the utxo information that Bitcoin Core adds to our PSBT before we sign it. We can safely used this information because if Bitcoin Core lies about the value of the inputs that we're spending then the signature we produce will also not be valid (it commits to the value being spent). When we're adding wallet inputs to "bump" the fees of a parent transaction we need to take the whole package into account when we verify the actual fee rate, which is why some internal methods were modified to return the package weight that was used as reference when `fundrawtransaction` was called. * Check that fundrawtransaction does not add more than 1 change output * Validate addresses and keys generated by bitcoin core When eclair manages private keys, make sure that we can re-compute addresses and keys generated by bitcoin core. * Add a separate configuration file for Eclair's onchain signer Eclair's onchain signer now has its own `eclair-signer.conf` configuration file in HOCON format. It includes BIP39 mnemonic codes and passphrase, a wallet name and a timestamp. When an `eclair-signer.conf` file is found, Eclair's API will return descriptors that can be imported into an empty watch-only Bitcoin Wallet. When wallet name in `eclair-signer.conf` matches the name of the Bitcoin Core wallet defined in `eclair.conf` (`eclair.bitcoind.wallet`), Eclair will bypass Bitcoin Core and sign onchain transactions directly. --- README.md | 5 +- docs/BitcoinCoreKeys.md | 96 ++++ docs/Configure.md | 2 +- docs/Guides.md | 1 + docs/release-notes/eclair-vnext.md | 8 + eclair-core/src/main/resources/reference.conf | 1 + .../main/scala/fr/acinq/eclair/Eclair.scala | 25 +- .../main/scala/fr/acinq/eclair/Setup.scala | 10 +- .../eclair/blockchain/OnChainWallet.scala | 11 +- .../rpc/BasicBitcoinJsonRPCClient.scala | 2 +- .../rpc/BatchingBitcoinJsonRPCClient.scala | 1 + .../bitcoind/rpc/BitcoinCoreClient.scala | 267 ++++++++-- .../bitcoind/rpc/BitcoinJsonRPCClient.scala | 1 + .../channel/fund/InteractiveTxBuilder.scala | 24 +- .../channel/publish/ReplaceableTxFunder.scala | 182 ++++--- .../publish/ReplaceableTxPrePublisher.scala | 27 +- .../keymanager/LocalOnchainKeyManager.scala | 165 ++++++ .../crypto/keymanager/OnchainKeyManager.scala | 40 ++ .../scala/fr/acinq/eclair/StartupSpec.scala | 2 +- .../blockchain/DummyOnChainWallet.scala | 31 +- .../bitcoind/BitcoinCoreClientSpec.scala | 471 ++++++++++++------ .../blockchain/bitcoind/BitcoindService.scala | 73 ++- .../blockchain/bitcoind/ZmqWatcherSpec.scala | 5 +- .../fee/BitcoinCoreFeeProviderSpec.scala | 3 +- .../channel/InteractiveTxBuilderSpec.scala | 42 +- .../publish/ReplaceableTxFunderSpec.scala | 6 +- .../publish/ReplaceableTxPublisherSpec.scala | 55 +- .../LocalChannelKeyManagerSpec.scala | 5 +- .../keymanager/LocalNodeKeyManagerSpec.scala | 6 +- .../LocalOnchainKeyManagerSpec.scala | 146 ++++++ .../integration/ChannelIntegrationSpec.scala | 15 +- .../eclair/integration/IntegrationSpec.scala | 4 +- .../integration/PaymentIntegrationSpec.scala | 2 +- .../basic/fixtures/MinimalNodeFixture.scala | 2 +- .../AnnouncementsBatchValidationSpec.scala | 7 +- .../acinq/eclair/api/handlers/OnChain.scala | 18 +- 36 files changed, 1413 insertions(+), 348 deletions(-) create mode 100644 docs/BitcoinCoreKeys.md create mode 100644 eclair-core/src/main/scala/fr/acinq/eclair/crypto/keymanager/LocalOnchainKeyManager.scala create mode 100644 eclair-core/src/main/scala/fr/acinq/eclair/crypto/keymanager/OnchainKeyManager.scala create mode 100644 eclair-core/src/test/scala/fr/acinq/eclair/crypto/keymanager/LocalOnchainKeyManagerSpec.scala diff --git a/README.md b/README.md index f90dd7b6bd..d54cf8b35d 100644 --- a/README.md +++ b/README.md @@ -162,7 +162,10 @@ limitdescendantcount=20 Setting these parameters lets you unblock long chains of unconfirmed channel funding transactions by using child-pays-for-parent (CPFP) to make them confirm. -With the default `bitcoind` parameters, if your node created a chain of 25 unconfirmed funding transactions with a low-feerate, you wouldn't be able to use CPFP to raise their fees because your CPFP transaction would likely be rejected by the rest of the network. +With the default `bitcoind` parameters, if your node created a chain of 25 unconfirmed funding transactions with a low-feerate, you wouldn't be able to use CPFP to raise their fees because your CPFP transaction would likely be rejected by +the rest of the network. + +You can also configure Eclair to manage Bitcoin Core's private keys, see our [guides](./docs/Guides.md) for more details. ### Java Environment Variables diff --git a/docs/BitcoinCoreKeys.md b/docs/BitcoinCoreKeys.md new file mode 100644 index 0000000000..9040f45c71 --- /dev/null +++ b/docs/BitcoinCoreKeys.md @@ -0,0 +1,96 @@ +# Using Eclair to manage your Bitcoin Core wallet's private keys + +You can configure Eclair to control (and never expose) the private keys of your Bitcoin Core wallet. This is very useful if your Bitcoin and Eclair nodes run on different machines for example, with a setup for the Bitcoin host that +is less secure than for Eclair (because it is shared among several services for example). + +Follow these steps to delegate onchain key management to eclair: + +1) Generate or import a BIP39 mnemonic code and passphrase + +You can use any BIP39-compatible tool, including most hardware wallets. + +2) Create an `eclair-signer.conf` configuration file add it to eclair's data directory + +A signer configuration file uses the HOCON format that we already use for `eclair.conf` and must include the following options: + + key | description +--------------------------|-------------------------------------------------------------------------- + eclair.signer.wallet | wallet name + eclair.signer.mnemonics | BIP39 mnemonic words + eclair.signer.passphrase | passphrase + eclair.signer.timestamp | wallet creation UNIX timestamp. Set to the current time for new wallets. + +This is an example of `eclair-signer.conf` configuration file: + +```hocon +{ + eclair { + signer { + wallet = "eclair" + mnemonics = "legal winner thank year wave sausage worth useful legal winner thank year wave sausage worth useful legal winner thank year wave sausage worth title" + passphrase = "" + timestamp = 1686055705 + } + } +} +``` + +You must set `eclair.signer.wallet` to a name that is different from your current Bitcoin Core wallet. + +3) Create an empty, descriptor-enabled, watch-only wallet in Bitcoin Core: + :warning: The name must match the one that you set in `eclair-signer.conf` (here we use "eclair") + +```shell +$ bitcoin-cli -named createwallet wallet_name=eclair disable_private_keys=true blank=true descriptors=true load_on_startup=true +``` + +4) Import public descriptors generated by Eclair + +`eclair-cli listdescriptors` will return public wallet descriptors in a format that is compatible with Bitcoin Core, and that you can import with `bitcoin-cli -rpcwallet=eclair importdescriptors` +For now, this descriptors follow the BIP84 standard (p2wpkh outputs). +This is an example of descriptors generated by Eclair: + +```json +[ + { + "desc": "wpkh([0d9250da/84h/1h/0h]tpubDDGF9PnrXww2h1mKNjKiXoqdDFGEcZGCZUNq7g26LdzKXKiE31RrFWsogPy1uMLrbG8ksQ8eJS6u6KFLjYUUSVJRuwmMD2SYCr8uG1TcRgM/0/*)#jz5n2pcp", + "internal": false, + "timestamp": 1686055705, + "active": true + }, + { + "desc": "wpkh([0d9250da/84h/1h/0h]tpubDDGF9PnrXww2h1mKNjKiXoqdDFGEcZGCZUNq7g26LdzKXKiE31RrFWsogPy1uMLrbG8ksQ8eJS6u6KFLjYUUSVJRuwmMD2SYCr8uG1TcRgM/1/*)#rk3jh5ge", + "internal": true, + "timestamp": 1686055705, + "active": true + } +] +``` + +You can combine the generation and import of descriptors with: + +```shell +$ eclair-cli getdescriptors | jq --raw-output -c | xargs -0 bitcoin-cli -rpcwallet=eclair importdescriptors +``` + +:warning: If you are restoring an existing `eclair-signer.conf` file with a timestamp that is fairly old, importing descriptors can take a long time, and your +Bitcoin Core node will not be usable until it's done + +5) Configure Eclair to handle private keys for this wallet + +Set `eclair.bitcoind.wallet` to the name of the wallet just created (`eclair` in the example above) and restart Eclair. + +You now have a Bitcoin Core watch-only wallet for which only your Eclair node can sign transactions. This Bitcoin Core wallet can +safely be copied to another Bitcoin Core node to monitor your onchain funds. + +You can also use `eclair-cli getmasterxpub` to get a BIP32 extended public key that you can import into any compatible Bitcoin wallet +to create a watch-only wallet (Electrum for example) that you can use to monitor your Bitcoin Core balance. + +:warning: this means that your Bitcoin Core wallet cannot send funds on its on (since it cannot access private keys to sign transactions). +To send funds onchain you must use `eclair-cli sendonchain`. + +:warning: to backup the private keys of this wallet you must either backup your mnemonic code and passphrase, or backup the `eclair-signer.conf` file in your eclair +directory (default is `~/.eclair`) along with your channels and node seed files. + +:warning: You can also initialise a backup onchain wallet with the same mnemonic code and passphrase (a hardware wallet for example), but be warned that using them may interfere with your node's operations (for example you may end up +double-spending funding transactions generated by your node). diff --git a/docs/Configure.md b/docs/Configure.md index 3e3626610f..c11375718b 100644 --- a/docs/Configure.md +++ b/docs/Configure.md @@ -4,7 +4,7 @@ * [Configuration file](#configuration-file) * [Changing the data directory](#changing-the-data-directory) - * [Splitting the configuration](#splitting-the-configuration) + * [Splitting the configuration](#splitting-the-configuration)\ * [Options reference](#options-reference) * [Customize features](#customize-features) * [Customize feerate tolerance](#customize-feerate-tolerance) diff --git a/docs/Guides.md b/docs/Guides.md index 6f7a19d0a9..85c07ed300 100644 --- a/docs/Guides.md +++ b/docs/Guides.md @@ -4,6 +4,7 @@ This section contains how-to guides for more advanced scenarios: * [Customize Logging](./Logging.md) * [Customize Features](./Features.md) +* [Manage Bitcoin Core's private keys](./BitcoinCoreKeys.md) * [Use Tor with Eclair](./Tor.md) * [Multipart Payments](./MultipartPayments.md) * [Trampoline Payments](./TrampolinePayments.md) diff --git a/docs/release-notes/eclair-vnext.md b/docs/release-notes/eclair-vnext.md index c0512fe5f9..91970c24d6 100644 --- a/docs/release-notes/eclair-vnext.md +++ b/docs/release-notes/eclair-vnext.md @@ -23,6 +23,14 @@ eclair.on-chain-fees.confirmation-priority { This configuration section replaces the previous `eclair.on-chain-fees.target-blocks` section. +### Managing Bitcoin Core wallet keys + +You can now create Bitcoin Core watch-only wallets and have Eclair manage their private keys: + +1. Create an empty, descriptors-enabled watch-only wallet in Bitcoin Core +2. Import wallet descriptors generated by Eclair into this wallet +3. Configure Eclair to use this wallet and set the `eclair.bitcoind.use-eclair-signer` to `true` + ### API changes - `bumpforceclose` can be used to make a force-close confirm faster, by spending the anchor output (#2743) diff --git a/eclair-core/src/main/resources/reference.conf b/eclair-core/src/main/resources/reference.conf index ecf7ae8452..ca6e96ea2a 100644 --- a/eclair-core/src/main/resources/reference.conf +++ b/eclair-core/src/main/resources/reference.conf @@ -40,6 +40,7 @@ eclair { // - ignore: eclair will leave these utxos locked and start startup-locked-utxos-behavior = "stop" final-pubkey-refresh-delay = 3 seconds + use-eclair-signer = false } node-alias = "eclair" diff --git a/eclair-core/src/main/scala/fr/acinq/eclair/Eclair.scala b/eclair-core/src/main/scala/fr/acinq/eclair/Eclair.scala index c3da91b4f6..045c35c4cc 100644 --- a/eclair-core/src/main/scala/fr/acinq/eclair/Eclair.scala +++ b/eclair-core/src/main/scala/fr/acinq/eclair/Eclair.scala @@ -32,6 +32,7 @@ import fr.acinq.eclair.blockchain.OnChainWallet.OnChainBalance import fr.acinq.eclair.blockchain.bitcoind.rpc.BitcoinCoreClient import fr.acinq.eclair.blockchain.bitcoind.rpc.BitcoinCoreClient.WalletTx import fr.acinq.eclair.blockchain.fee.{ConfirmationTarget, FeeratePerByte, FeeratePerKw} +import fr.acinq.eclair.blockchain.bitcoind.rpc.BitcoinCoreClient.{Descriptors, WalletTx} import fr.acinq.eclair.channel._ import fr.acinq.eclair.crypto.Sphinx import fr.acinq.eclair.db.AuditDb.{NetworkFee, Stats} @@ -180,6 +181,10 @@ trait Eclair { def payOfferBlocking(offer: Offer, amount: MilliSatoshi, quantity: Long, externalId_opt: Option[String] = None, maxAttempts_opt: Option[Int] = None, maxFeeFlat_opt: Option[Satoshi] = None, maxFeePct_opt: Option[Double] = None, pathFindingExperimentName_opt: Option[String] = None, connectDirectly: Boolean = false)(implicit timeout: Timeout): Future[PaymentEvent] + def getOnchainMasterPubKey(account: Long): String + + def getDescriptors(account: Long): Descriptors + def stop(): Future[Unit] } @@ -353,8 +358,16 @@ class EclairImpl(appKit: Kit) extends Eclair with Logging { } override def sendOnChain(address: String, amount: Satoshi, confirmationTarget: Long): Future[ByteVector32] = { + val feeRate = if (confirmationTarget < 3) appKit.nodeParams.currentFeerates.fast + else if (confirmationTarget > 6) appKit.nodeParams.currentFeerates.slow + else appKit.nodeParams.currentFeerates.medium + appKit.wallet match { - case w: BitcoinCoreClient => w.sendToAddress(address, amount, confirmationTarget) + case w: BitcoinCoreClient => + addressToPublicKeyScript(appKit.nodeParams.chainHash, address) match { + case Right(pubkeyScript) => w.sendToPubkeyScript(pubkeyScript, amount, feeRate) + case Left(failure) => Future.failed(new IllegalArgumentException(s"invalid address ($failure)")) + } case _ => Future.failed(new IllegalArgumentException("this call is only available with a bitcoin core backend")) } } @@ -717,6 +730,16 @@ class EclairImpl(appKit: Kit) extends Eclair with Logging { payOfferInternal(offer, amount, quantity, externalId_opt, maxAttempts_opt, maxFeeFlat_opt, maxFeePct_opt, pathFindingExperimentName_opt, connectDirectly, blocking = true).mapTo[PaymentEvent] } + override def getDescriptors(account: Long): Descriptors = appKit.wallet match { + case bitcoinCoreClient: BitcoinCoreClient if bitcoinCoreClient.onchainKeyManager_opt.isDefined => bitcoinCoreClient.onchainKeyManager_opt.get.getDescriptors(account) + case _ => throw new RuntimeException("onchain seed is not configured") + } + + override def getOnchainMasterPubKey(account: Long): String = appKit.wallet match { + case bitcoinCoreClient: BitcoinCoreClient if bitcoinCoreClient.onchainKeyManager_opt.isDefined => bitcoinCoreClient.onchainKeyManager_opt.get.getOnchainMasterPubKey(account) + case _ => throw new RuntimeException("onchain seed is not configured") + } + override def stop(): Future[Unit] = { // README: do not make this smarter or more complex ! // eclair can simply and cleanly be stopped by killing its process without fear of losing data, payments, ... and it should remain this way. diff --git a/eclair-core/src/main/scala/fr/acinq/eclair/Setup.scala b/eclair-core/src/main/scala/fr/acinq/eclair/Setup.scala index 625de3e30a..e0114e403b 100644 --- a/eclair-core/src/main/scala/fr/acinq/eclair/Setup.scala +++ b/eclair-core/src/main/scala/fr/acinq/eclair/Setup.scala @@ -34,7 +34,7 @@ import fr.acinq.eclair.blockchain.fee._ import fr.acinq.eclair.channel.Register import fr.acinq.eclair.channel.fsm.Channel import fr.acinq.eclair.crypto.WeakEntropyPool -import fr.acinq.eclair.crypto.keymanager.{LocalChannelKeyManager, LocalNodeKeyManager} +import fr.acinq.eclair.crypto.keymanager.{LocalChannelKeyManager, LocalNodeKeyManager, LocalOnchainKeyManager} import fr.acinq.eclair.db.Databases.FileBackup import fr.acinq.eclair.db.FileBackupHandler.FileBackupParams import fr.acinq.eclair.db.{Databases, DbEventHandler, FileBackupHandler} @@ -75,7 +75,8 @@ import scala.util.{Failure, Success} class Setup(val datadir: File, pluginParams: Seq[PluginParams], seeds_opt: Option[Seeds] = None, - db: Option[Databases] = None)(implicit system: ActorSystem) extends Logging { + db: Option[Databases] = None, + onchainKeyManager_opt: Option[LocalOnchainKeyManager] = None)(implicit system: ActorSystem) extends Logging { implicit val timeout: Timeout = Timeout(30 seconds) implicit val formats: org.json4s.Formats = org.json4s.DefaultFormats @@ -221,6 +222,8 @@ class Setup(val datadir: File, feeProvider = nodeParams.chainHash match { case Block.RegtestGenesisBlock.hash => FallbackFeeProvider(ConstantFeeProvider(defaultFeerates) :: Nil, minFeeratePerByte) + case Block.SignetGenesisBlock.hash => + FallbackFeeProvider(ConstantFeeProvider(defaultFeerates) :: Nil, minFeeratePerByte) case _ => FallbackFeeProvider(SmoothFeeProvider(BitcoinCoreFeeProvider(bitcoin, defaultFeerates), smoothFeerateWindow) :: Nil, minFeeratePerByte) } @@ -244,7 +247,7 @@ class Setup(val datadir: File, finalPubkey = new AtomicReference[PublicKey](null) pubkeyRefreshDelay = FiniteDuration(config.getDuration("bitcoind.final-pubkey-refresh-delay").getSeconds, TimeUnit.SECONDS) - bitcoinClient = new BitcoinCoreClient(bitcoin) with OnchainPubkeyCache { + bitcoinClient = new BitcoinCoreClient(bitcoin, onchainKeyManager_opt.orElse(LocalOnchainKeyManager.load(datadir, nodeParams.chainHash))) with OnchainPubkeyCache { val refresher: typed.ActorRef[OnchainPubkeyRefresher.Command] = system.spawn(Behaviors.supervise(OnchainPubkeyRefresher(this, finalPubkey, pubkeyRefreshDelay)).onFailure(typed.SupervisorStrategy.restart), name = "onchain-address-manager") override def getP2wpkhPubkey(renew: Boolean): PublicKey = { @@ -253,6 +256,7 @@ class Setup(val datadir: File, key } } + _ = if (bitcoinClient.useEclairSigner) logger.info("using eclair to sign bitcoin core transactions") initialPubkey <- bitcoinClient.getP2wpkhPubkey() _ = finalPubkey.set(initialPubkey) diff --git a/eclair-core/src/main/scala/fr/acinq/eclair/blockchain/OnChainWallet.scala b/eclair-core/src/main/scala/fr/acinq/eclair/blockchain/OnChainWallet.scala index 031df91079..47c5bb0c0d 100644 --- a/eclair-core/src/main/scala/fr/acinq/eclair/blockchain/OnChainWallet.scala +++ b/eclair-core/src/main/scala/fr/acinq/eclair/blockchain/OnChainWallet.scala @@ -16,8 +16,10 @@ package fr.acinq.eclair.blockchain +import fr.acinq.bitcoin.psbt.Psbt import fr.acinq.bitcoin.scalacompat.Crypto.PublicKey import fr.acinq.bitcoin.scalacompat.{ByteVector32, OutPoint, Satoshi, Transaction} +import fr.acinq.eclair.blockchain.bitcoind.rpc.BitcoinCoreClient.ProcessPsbtResponse import fr.acinq.eclair.blockchain.fee.FeeratePerKw import scodec.bits.ByteVector @@ -35,8 +37,10 @@ trait OnChainChannelFunder { /** Fund the provided transaction by adding inputs (and a change output if necessary). */ def fundTransaction(tx: Transaction, feeRate: FeeratePerKw, replaceable: Boolean, externalInputsWeight: Map[OutPoint, Long] = Map.empty)(implicit ec: ExecutionContext): Future[FundTransactionResponse] - /** Sign the wallet inputs of the provided transaction. */ - def signTransaction(tx: Transaction, allowIncomplete: Boolean)(implicit ec: ExecutionContext): Future[SignTransactionResponse] + /** + * sign a PSBT. Result may be partially signed: only inputs known to our bitcoin core wallet will be signed + */ + def signPsbt(psbt: Psbt, ourInputs: Seq[Int], ourOutputs: Seq[Int])(implicit ec: ExecutionContext): Future[ProcessPsbtResponse] /** * Publish a transaction on the bitcoin network. @@ -121,7 +125,4 @@ object OnChainWallet { final case class FundTransactionResponse(tx: Transaction, fee: Satoshi, changePosition: Option[Int]) { val amountIn: Satoshi = fee + tx.txOut.map(_.amount).sum } - - final case class SignTransactionResponse(tx: Transaction, complete: Boolean) - } diff --git a/eclair-core/src/main/scala/fr/acinq/eclair/blockchain/bitcoind/rpc/BasicBitcoinJsonRPCClient.scala b/eclair-core/src/main/scala/fr/acinq/eclair/blockchain/bitcoind/rpc/BasicBitcoinJsonRPCClient.scala index 4886d66ac4..d9c1731c30 100644 --- a/eclair-core/src/main/scala/fr/acinq/eclair/blockchain/bitcoind/rpc/BasicBitcoinJsonRPCClient.scala +++ b/eclair-core/src/main/scala/fr/acinq/eclair/blockchain/bitcoind/rpc/BasicBitcoinJsonRPCClient.scala @@ -32,7 +32,7 @@ import java.util.concurrent.atomic.AtomicReference import scala.concurrent.{ExecutionContext, Future} import scala.util.{Failure, Success, Try} -class BasicBitcoinJsonRPCClient(rpcAuthMethod: BitcoinJsonRPCAuthMethod, host: String = "127.0.0.1", port: Int = 8332, ssl: Boolean = false, wallet: Option[String] = None)(implicit sb: SttpBackend[Future, _]) extends BitcoinJsonRPCClient { +class BasicBitcoinJsonRPCClient(rpcAuthMethod: BitcoinJsonRPCAuthMethod, host: String = "127.0.0.1", port: Int = 8332, ssl: Boolean = false, override val wallet: Option[String] = None)(implicit sb: SttpBackend[Future, _]) extends BitcoinJsonRPCClient { implicit val formats: Formats = DefaultFormats.withBigDecimal + ByteVector32Serializer + ByteVector32KmpSerializer diff --git a/eclair-core/src/main/scala/fr/acinq/eclair/blockchain/bitcoind/rpc/BatchingBitcoinJsonRPCClient.scala b/eclair-core/src/main/scala/fr/acinq/eclair/blockchain/bitcoind/rpc/BatchingBitcoinJsonRPCClient.scala index 16da82573b..6e36041234 100644 --- a/eclair-core/src/main/scala/fr/acinq/eclair/blockchain/bitcoind/rpc/BatchingBitcoinJsonRPCClient.scala +++ b/eclair-core/src/main/scala/fr/acinq/eclair/blockchain/bitcoind/rpc/BatchingBitcoinJsonRPCClient.scala @@ -27,6 +27,7 @@ import scala.concurrent.duration._ import scala.concurrent.{ExecutionContext, Future} class BatchingBitcoinJsonRPCClient(rpcClient: BasicBitcoinJsonRPCClient)(implicit system: ActorSystem, ec: ExecutionContext) extends BitcoinJsonRPCClient { + override def wallet: Option[String] = rpcClient.wallet implicit val timeout: Timeout = Timeout(1 hour) diff --git a/eclair-core/src/main/scala/fr/acinq/eclair/blockchain/bitcoind/rpc/BitcoinCoreClient.scala b/eclair-core/src/main/scala/fr/acinq/eclair/blockchain/bitcoind/rpc/BitcoinCoreClient.scala index 2df951d9ba..1edd2ebe1b 100644 --- a/eclair-core/src/main/scala/fr/acinq/eclair/blockchain/bitcoind/rpc/BitcoinCoreClient.scala +++ b/eclair-core/src/main/scala/fr/acinq/eclair/blockchain/bitcoind/rpc/BitcoinCoreClient.scala @@ -16,14 +16,16 @@ package fr.acinq.eclair.blockchain.bitcoind.rpc +import fr.acinq.bitcoin.psbt.{Psbt, UpdateFailure} import fr.acinq.bitcoin.scalacompat.Crypto.PublicKey import fr.acinq.bitcoin.scalacompat._ -import fr.acinq.bitcoin.{Bech32, Block} +import fr.acinq.bitcoin.{Bech32, Block, SigHash} import fr.acinq.eclair.ShortChannelId.coordinates import fr.acinq.eclair.blockchain.OnChainWallet -import fr.acinq.eclair.blockchain.OnChainWallet.{FundTransactionResponse, MakeFundingTxResponse, OnChainBalance, SignTransactionResponse} +import fr.acinq.eclair.blockchain.OnChainWallet.{FundTransactionResponse, MakeFundingTxResponse, OnChainBalance} import fr.acinq.eclair.blockchain.bitcoind.ZmqWatcher.{GetTxWithMetaResponse, UtxoStatus, ValidateResult} import fr.acinq.eclair.blockchain.fee.{FeeratePerKB, FeeratePerKw} +import fr.acinq.eclair.crypto.keymanager.OnchainKeyManager import fr.acinq.eclair.transactions.Transactions import fr.acinq.eclair.wire.protocol.ChannelAnnouncement import fr.acinq.eclair.{BlockHeight, TimestampSecond, TxCoordinates} @@ -32,6 +34,7 @@ import org.json4s.Formats import org.json4s.JsonAST._ import scodec.bits.ByteVector +import java.util.Base64 import scala.concurrent.{ExecutionContext, Future} import scala.jdk.CollectionConverters.ListHasAsScala import scala.util.{Failure, Success, Try} @@ -42,13 +45,19 @@ import scala.util.{Failure, Success, Try} /** * The Bitcoin Core client provides some high-level utility methods to interact with Bitcoin Core. + * + * @param rpcClient bitcoin core JSON rpc client + * @param onchainKeyManager_opt optional onchain key manager. If provided and its wallet name matcher our rpcClient wallet's name, it will be used to sign transactions (it is assumed that bitcoin + * core uses a watch-only wallet with descriptors generated by Eclair with this onchain key manager) */ -class BitcoinCoreClient(val rpcClient: BitcoinJsonRPCClient) extends OnChainWallet with Logging { +class BitcoinCoreClient(val rpcClient: BitcoinJsonRPCClient, val onchainKeyManager_opt: Option[OnchainKeyManager] = None) extends OnChainWallet with Logging { import BitcoinCoreClient._ implicit val formats: Formats = org.json4s.DefaultFormats + val useEclairSigner = onchainKeyManager_opt.exists(m => rpcClient.wallet.contains(m.wallet)) + //------------------------- TRANSACTIONS -------------------------// def getTransaction(txid: ByteVector32)(implicit ec: ExecutionContext): Future[Transaction] = @@ -220,15 +229,28 @@ class BitcoinCoreClient(val rpcClient: BitcoinJsonRPCClient) extends OnChainWall } //------------------------- FUNDING -------------------------// - def fundTransaction(tx: Transaction, options: FundTransactionOptions)(implicit ec: ExecutionContext): Future[FundTransactionResponse] = { - rpcClient.invoke("fundrawtransaction", tx.toString(), options).map(json => { + rpcClient.invoke("fundrawtransaction", tx.toString(), options).flatMap(json => { val JString(hex) = json \ "hex" val JInt(changePos) = json \ "changepos" val JDecimal(fee) = json \ "fee" val fundedTx = Transaction.read(hex) val changePos_opt = if (changePos >= 0) Some(changePos.intValue) else None - FundTransactionResponse(fundedTx, toSatoshi(fee), changePos_opt) + + val walletInputs = fundedTx.txIn.map(_.outPoint).toSet -- tx.txIn.map(_.outPoint).toSet + val addedOutputs = fundedTx.txOut.size - tx.txOut.size + Try { + require(addedOutputs <= 1, "more than one change output added") + require(addedOutputs == 0 || changePos >= 0, "change output added, but position not returned") + require(changePos < 0 || !tx.txOut.contains(fundedTx.txOut(changePos.intValue)), "existing output returned as change output") + require(options.changePosition.isEmpty || changePos == options.changePosition.get || changePos == -1, "change position added at wrong position") + + FundTransactionResponse(fundedTx, toSatoshi(fee), changePos_opt) + } + match { + case Success(response) => Future.successful(response) + case Failure(error) => unlockOutpoints(walletInputs.toSeq).flatMap(_ => Future.failed(error)) + } }) } @@ -236,25 +258,92 @@ class BitcoinCoreClient(val rpcClient: BitcoinJsonRPCClient) extends OnChainWall fundTransaction(tx, FundTransactionOptions(feeRate, replaceable, inputWeights = externalInputsWeight.map { case (outpoint, weight) => InputWeight(outpoint, weight) }.toSeq)) } + def processPsbt(psbt: Psbt, sign: Boolean = true, sighashType: Option[Int] = None)(implicit ec: ExecutionContext): Future[ProcessPsbtResponse] = { + // we use an explicit map here because this RPC calls takes a String for the "sighashtype" parameter with an explicitly limited list of valid values + val sighashStrings = Map( + SigHash.SIGHASH_DEFAULT -> "DEFAULT", + SigHash.SIGHASH_ALL -> "ALL", + SigHash.SIGHASH_NONE -> "NONE", + SigHash.SIGHASH_SINGLE -> "SINGLE", + (SigHash.SIGHASH_ALL | SigHash.SIGHASH_ANYONECANPAY) -> "ALL|ANYONECANPAY", + (SigHash.SIGHASH_NONE | SigHash.SIGHASH_ANYONECANPAY) -> "NONE|ANYONECANPAY", + (SigHash.SIGHASH_SINGLE | SigHash.SIGHASH_ANYONECANPAY) -> "SINGLE|ANYONECANPAY") + val sighash = sighashType.map(s => sighashStrings.getOrElse(s, throw new IllegalArgumentException(s"invalid sighash flag ${sighashType}"))) + val encoded = Base64.getEncoder.encodeToString(Psbt.write(psbt).toByteArray) + val params = Seq(encoded, sign) ++ sighash.toSeq + rpcClient.invoke("walletprocesspsbt", params: _*).map(json => { + val JString(base64) = json \ "psbt" + val JBool(complete) = json \ "complete" + val decoded = Psbt.read(Base64.getDecoder.decode(base64)) + require(decoded.isRight, s"cannot decode psbt from $base64") + ProcessPsbtResponse(decoded.getRight, complete) + }) + } + + def utxoUpdatePsbt(psbt: Psbt)(implicit ec: ExecutionContext): Future[Psbt] = { + val encoded = Base64.getEncoder.encodeToString(Psbt.write(psbt).toByteArray) + + rpcClient.invoke("utxoupdatepsbt", encoded).map(json => { + val JString(base64) = json + val bin = Base64.getDecoder.decode(base64) + val decoded = Psbt.read(bin) + require(decoded.isRight, s"cannot decode psbt from $base64") + val psbt = decoded.getRight + psbt + }) + } + + private def unlockIfFails[T](txid: ByteVector32, locked: Seq[OutPoint])(f: => Future[T])(implicit ec: ExecutionContext): Future[T] = { + // if signature fails (e.g. because wallet is encrypted) we need to unlock the utxos + f.recoverWith { case _ => + unlockOutpoints(locked) + .recover { case t: Throwable => // no-op, just add a log in case of failure + logger.warn(s"Cannot unlock failed transaction's UTXOs txid=${txid}", t) + t + } + .flatMap(_ => f) // return signTransaction error + .recoverWith { case _ => f } // return signTransaction error + } + } + def makeFundingTx(pubkeyScript: ByteVector, amount: Satoshi, targetFeerate: FeeratePerKw)(implicit ec: ExecutionContext): Future[MakeFundingTxResponse] = { + import KotlinUtils._ + + def sign(tx: Transaction, fees: Satoshi, requestedFeeRate: FeeratePerKw): Future[MakeFundingTxResponse] = { + val outputIndex = Transactions.findPubKeyScriptIndex(tx, pubkeyScript) getOrElse { + throw new RuntimeException(s"cannot find expected funding output") + } + val ourInputs = tx.txIn.indices.toList // all inputs are supposed to be ours + val ourOutputs = tx.txOut.indices.toList filterNot (_ == outputIndex) // all outputs except for the funding output are supposed to be ours + val psbt = new Psbt(tx) + for { + signed <- signPsbt(psbt, ourInputs, ourOutputs) + extracted = signed.psbt.extract() + _ = require(extracted.isRight, s"signing psbt failed with ${extracted.getLeft}") + actualFees = kmp2scala(signed.psbt.computeFees()) + _ = require(actualFees == fees, s"actual funding fees $actualFees do not match bitcoin core fee $fees") + fundingTx = kmp2scala(extracted.getRight) + actualFeerate = FeeratePerKw((actualFees * 1000) / fundingTx.weight()) + maxFeerate = requestedFeeRate + requestedFeeRate / 2 + _ = require(actualFeerate < maxFeerate, s"actual feerate $actualFeerate is more than 50% above requested fee rate $targetFeerate") + _ = logger.debug(s"created funding txid=${extracted.getRight.txid} outputIndex=$outputIndex fee=${fees}") + } yield MakeFundingTxResponse(fundingTx, outputIndex, fees) + } + val partialFundingTx = Transaction( version = 2, txIn = Seq.empty[TxIn], txOut = TxOut(amount, pubkeyScript) :: Nil, lockTime = 0) + for { + // TODO: we should check that mempoolMinFee is not dangerously high feerate <- mempoolMinFee().map(minFee => FeeratePerKw(minFee).max(targetFeerate)) // we ask bitcoin core to add inputs to the funding tx, and use the specified change address - fundTxResponse <- fundTransaction(partialFundingTx, FundTransactionOptions(feerate)) - // now let's sign the funding tx - SignTransactionResponse(fundingTx, true) <- signTransactionOrUnlock(fundTxResponse.tx) - // there will probably be a change output, so we need to find which output is ours - outputIndex <- Transactions.findPubKeyScriptIndex(fundingTx, pubkeyScript) match { - case Right(outputIndex) => Future.successful(outputIndex) - case Left(skipped) => Future.failed(new RuntimeException(skipped.toString)) - } - _ = logger.debug(s"created funding txid=${fundingTx.txid} outputIndex=$outputIndex fee=${fundTxResponse.fee}") - } yield MakeFundingTxResponse(fundingTx, outputIndex, fundTxResponse.fee) + FundTransactionResponse(tx, fee, _) <- fundTransaction(partialFundingTx, FundTransactionOptions(feerate)) + lockedUtxos = tx.txIn.map(_.outPoint) + signedTx <- unlockIfFails(tx.txid, lockedUtxos)(sign(tx, fee, feerate)) + } yield signedTx } def commit(tx: Transaction)(implicit ec: ExecutionContext): Future[Boolean] = publishTransaction(tx).transformWith { @@ -278,6 +367,8 @@ class BitcoinCoreClient(val rpcClient: BitcoinJsonRPCClient) extends OnChainWall * @param targetFeerate feerate to apply to the package of unconfirmed transactions. */ def cpfp(outpoints: Set[OutPoint], targetFeerate: FeeratePerKw)(implicit ec: ExecutionContext): Future[Transaction] = { + import KotlinUtils._ + getMempoolPackage(outpoints.map(_.txid)).transformWith { case Failure(ex) => Future.failed(new IllegalArgumentException("unable to analyze mempool package: some transactions could not be found in your mempool", ex)) case Success(mempoolPackage) => @@ -303,9 +394,13 @@ class BitcoinCoreClient(val rpcClient: BitcoinJsonRPCClient) extends OnChainWall Future.failed(new IllegalArgumentException("input amount is not sufficient to cover the target feerate")) } else { val unsignedTx = Transaction(2, outpoints.toSeq.map(o => TxIn(o, Seq.empty, 0)), Seq(TxOut(amountIn - missingFees, Script.pay2wpkh(changePubkeyHash))), 0) - signTransaction(unsignedTx, Nil).transformWith { + signPsbt(new Psbt(unsignedTx), unsignedTx.txIn.indices, unsignedTx.txOut.indices).transformWith { case Failure(ex) => Future.failed(new IllegalArgumentException("tx signing failed: some inputs don't belong to our wallet", ex)) - case Success(signedTx) => publishTransaction(signedTx.tx).map(_ => signedTx.tx) + case Success(response) => + response.extractFinalTx match { + case Left(error) => Future.failed(new IllegalArgumentException("tx signing failed: some inputs don't belong to our wallet", new RuntimeException(error.toString))) + case Right(tx) => publishTransaction(tx).map(_ => tx) + } } } } @@ -344,41 +439,23 @@ class BitcoinCoreClient(val rpcClient: BitcoinJsonRPCClient) extends OnChainWall //------------------------- SIGNING -------------------------// - def signTransaction(tx: Transaction)(implicit ec: ExecutionContext): Future[SignTransactionResponse] = signTransaction(tx, Nil) + def signPsbt(psbt: Psbt, ourInputs: Seq[Int], ourOutputs: Seq[Int])(implicit ec: ExecutionContext): Future[ProcessPsbtResponse] = { - def signTransaction(tx: Transaction, allowIncomplete: Boolean)(implicit ec: ExecutionContext): Future[SignTransactionResponse] = signTransaction(tx, Nil, allowIncomplete) - - def signTransaction(tx: Transaction, previousTxs: Seq[PreviousTx], allowIncomplete: Boolean = false)(implicit ec: ExecutionContext): Future[SignTransactionResponse] = { - rpcClient.invoke("signrawtransactionwithwallet", tx.toString(), previousTxs).map(json => { - val JString(hex) = json \ "hex" - val JBool(complete) = json \ "complete" - if (!complete && !allowIncomplete) { - val JArray(errors) = json \ "errors" - val message = errors.map(error => { - val JString(txid) = error \ "txid" - val JInt(vout) = error \ "vout" - val JString(scriptSig) = error \ "scriptSig" - val JString(message) = error \ "error" - s"txid=$txid vout=$vout scriptSig=$scriptSig error=$message" - }).mkString(", ") - throw JsonRPCError(Error(-1, message)) + def sign(psbt: Psbt): Future[ProcessPsbtResponse] = if (useEclairSigner) { + val onchainKeyManager = onchainKeyManager_opt.getOrElse(throw new RuntimeException("We are configured to use an eclair signer has not been loaded")) // this should not be possible + Try(onchainKeyManager.signPsbt(psbt, ourInputs, ourOutputs)) match { + case Success(signedPsbt) => Future.successful(ProcessPsbtResponse(signedPsbt, signedPsbt.extract().isRight)) + case Failure(error) => Future.failed(error) } - SignTransactionResponse(Transaction.read(hex), complete) - }) - } - - private def signTransactionOrUnlock(tx: Transaction)(implicit ec: ExecutionContext): Future[SignTransactionResponse] = { - val f = signTransaction(tx) - // if signature fails (e.g. because wallet is encrypted) we need to unlock the utxos - f.recoverWith { case _ => - unlockOutpoints(tx.txIn.map(_.outPoint)) - .recover { case t: Throwable => // no-op, just add a log in case of failure - logger.warn(s"Cannot unlock failed transaction's UTXOs txid=${tx.txid}", t) - t - } - .flatMap(_ => f) // return signTransaction error - .recoverWith { case _ => f } // return signTransaction error + } else { + processPsbt(psbt, sign = true) } + + for { + updated <- utxoUpdatePsbt(psbt) + filled <- processPsbt(updated, sign = false) // called with sign=false to fill input and output HD paths + signed <- sign(filled.psbt) + } yield signed } //------------------------- PUBLISHING -------------------------// @@ -456,14 +533,34 @@ class BitcoinCoreClient(val rpcClient: BitcoinJsonRPCClient) extends OnChainWall OnChainBalance(toSatoshi(confirmed), toSatoshi(unconfirmed)) }) + private def extractPublicKey(address: String)(implicit ec: ExecutionContext): Future[PublicKey] = { + for { + addressInfo <- rpcClient.invoke("getaddressinfo", address) + JString(keyPath) = addressInfo \ "hdkeypath" + JString(rawKey) = addressInfo \ "pubkey" + } yield { + val extracted = PublicKey(ByteVector.fromValidHex(rawKey)) + // check that when we manage private keys we can re-compute the public key we got from bitcoin core + // and that the address and public key match + if (useEclairSigner) { + val onchainKeyManager = onchainKeyManager_opt.getOrElse(throw new RuntimeException("We are configured to use an eclair signer has not been loaded")) // this should not be possible + val keyPath1 = keyPath.replace('h', '\'') // our bitcoin lib expects a ' suffix for hardened indexes and does not yet accept the h suffix + val computed = onchainKeyManager.getPublicKey(DeterministicWallet.KeyPath(keyPath1)) + require(computed == (extracted, address), "cannot recompute pubkey generated by bitcoin core") + } + extracted + } + } + def getReceiveAddress(label: String)(implicit ec: ExecutionContext): Future[String] = for { JString(address) <- rpcClient.invoke("getnewaddress", label) + _ <- extractPublicKey(address) } yield address def getP2wpkhPubkey()(implicit ec: ExecutionContext): Future[Crypto.PublicKey] = for { - address <- rpcClient.invoke("getnewaddress", "", "bech32") - JString(rawKey) <- rpcClient.invoke("getaddressinfo", address).map(_ \ "pubkey") - } yield PublicKey(ByteVector.fromValidHex(rawKey)) + JString(address) <- rpcClient.invoke("getnewaddress", "", "bech32") + pubKey <- extractPublicKey(address) + } yield pubKey /** * @return the public key hash of a bech32 raw change address. @@ -476,6 +573,41 @@ class BitcoinCoreClient(val rpcClient: BitcoinJsonRPCClient) extends OnChainWall } } + + /** + * Asks Bitcoin Core to fund and broadcsat a tx that sends funds to a given pubkey script + * If the current wallet uses Eclair to sign transaction, then we'll use our onchain key manager to sign the transaction, + * with the following assumptions: + * - all inputs belong to us + * - all outputs except for the one that sends to `pubkeyScript` belong to us + * + * @param pubkeyScript public key script to sent funds to + * @param amount amount to send + * @param feeratePerKw fee rate + * @return the txid of the sending tx. + */ + def sendToPubkeyScript(pubkeyScript: ByteVector, amount: Satoshi, feeratePerKw: FeeratePerKw)(implicit ec: ExecutionContext): Future[ByteVector32] = { + import KotlinUtils._ + + val theirOutput = TxOut(amount, pubkeyScript) + val tx = Transaction(version = 2, txIn = Nil, txOut = theirOutput :: Nil, lockTime = 0) + for { + fundedTx <- fundTransaction(tx, feeratePerKw, true) + lockedOutputs = fundedTx.tx.txIn.map(_.outPoint) + theirOutputPos = fundedTx.tx.txOut.indexOf(theirOutput) + signedPsbt <- unlockIfFails(fundedTx.tx.txid, lockedOutputs)(signPsbt(new Psbt(fundedTx.tx), fundedTx.tx.txIn.indices, fundedTx.tx.txOut.indices.filterNot(_ == theirOutputPos))) + signedTx = signedPsbt.finalTx + actualFees = kmp2scala(signedPsbt.psbt.computeFees()) + actualFeerate = FeeratePerKw((actualFees * 1000) / signedTx.weight()) + maxFeerate = feeratePerKw + feeratePerKw / 2 + _ = require(actualFeerate < maxFeerate, s"actual fee rate $actualFeerate is more than 50% above requested fee rate $feeratePerKw") + txid <- publishTransaction(signedTx) + } yield txid + } + + def sendToPubkeyScript(pubkeyScript: Seq[ScriptElt], amount: Satoshi, feeratePerKw: FeeratePerKw)(implicit ec: ExecutionContext): Future[ByteVector32] = sendToPubkeyScript(Script.write(pubkeyScript), amount, feeratePerKw) + + // calls Bitcoin Core's sendtoaddress RPC call directly. Will fail if wallet is using an external signer def sendToAddress(address: String, amount: Satoshi, confirmationTarget: Long)(implicit ec: ExecutionContext): Future[ByteVector32] = { rpcClient.invoke( "sendtoaddress", @@ -600,6 +732,32 @@ object BitcoinCoreClient { } } + case class ProcessPsbtResponse(psbt: Psbt, complete: Boolean) { + + import KotlinUtils._ + + // Extract a fully signed transaction from `psbt` + // If the transaction is just partially signed, this method will fail and you must call extractPartiallySignedTx instead + def extractFinalTx: Either[UpdateFailure, Transaction] = { + val extracted = psbt.extract() + if (extracted.isLeft) Left(extracted.getLeft) else Right(extracted.getRight) + } + + // Extract a partially signed transaction from `psbt` + def extractPartiallySignedTx: Transaction = { + var partiallySignedTx: Transaction = psbt.getGlobal.getTx + for (i <- 0 until psbt.getInputs.size()) { + val scriptWitness = psbt.getInputs.get(i).getScriptWitness + if (scriptWitness != null) { + partiallySignedTx = partiallySignedTx.updateWitness(i, scriptWitness) + } + } + partiallySignedTx + } + + def finalTx = extractFinalTx.getOrElse(throw new RuntimeException("cannot extract transaction from psbt")) + } + case class PreviousTx(txid: ByteVector32, vout: Long, scriptPubKey: String, redeemScript: String, witnessScript: String, amount: BigDecimal) object PreviousTx { @@ -638,4 +796,7 @@ object BitcoinCoreClient { def toSatoshi(btcAmount: BigDecimal): Satoshi = Satoshi(btcAmount.bigDecimal.scaleByPowerOfTen(8).longValue) + case class Descriptor(desc: String, internal: Boolean = false, timestamp: Either[String, Long] = Left("now"), active: Boolean = true) + + case class Descriptors(wallet_name: String, descriptors: Seq[Descriptor]) } \ No newline at end of file diff --git a/eclair-core/src/main/scala/fr/acinq/eclair/blockchain/bitcoind/rpc/BitcoinJsonRPCClient.scala b/eclair-core/src/main/scala/fr/acinq/eclair/blockchain/bitcoind/rpc/BitcoinJsonRPCClient.scala index 5373cc7947..63468d9ca9 100644 --- a/eclair-core/src/main/scala/fr/acinq/eclair/blockchain/bitcoind/rpc/BitcoinJsonRPCClient.scala +++ b/eclair-core/src/main/scala/fr/acinq/eclair/blockchain/bitcoind/rpc/BitcoinJsonRPCClient.scala @@ -22,6 +22,7 @@ import java.io.IOException import scala.concurrent.{ExecutionContext, Future} trait BitcoinJsonRPCClient { + def wallet: Option[String] def invoke(method: String, params: Any*)(implicit ec: ExecutionContext): Future[JValue] diff --git a/eclair-core/src/main/scala/fr/acinq/eclair/channel/fund/InteractiveTxBuilder.scala b/eclair-core/src/main/scala/fr/acinq/eclair/channel/fund/InteractiveTxBuilder.scala index 572811a86d..f20be9cc18 100644 --- a/eclair-core/src/main/scala/fr/acinq/eclair/channel/fund/InteractiveTxBuilder.scala +++ b/eclair-core/src/main/scala/fr/acinq/eclair/channel/fund/InteractiveTxBuilder.scala @@ -20,10 +20,10 @@ import akka.actor.typed.scaladsl.{ActorContext, Behaviors, StashBuffer} import akka.actor.typed.{ActorRef, Behavior} import akka.event.LoggingAdapter import fr.acinq.bitcoin.ScriptFlags +import fr.acinq.bitcoin.psbt.Psbt import fr.acinq.bitcoin.scalacompat.Crypto.PublicKey import fr.acinq.bitcoin.scalacompat.{ByteVector32, ByteVector64, LexicographicalOrdering, OutPoint, Satoshi, SatoshiLong, Script, ScriptWitness, Transaction, TxIn, TxOut} import fr.acinq.eclair.blockchain.OnChainChannelFunder -import fr.acinq.eclair.blockchain.OnChainWallet.SignTransactionResponse import fr.acinq.eclair.blockchain.fee.FeeratePerKw import fr.acinq.eclair.channel.Helpers.Closing.MutualClose import fr.acinq.eclair.channel.Helpers.Funding @@ -766,16 +766,30 @@ private class InteractiveTxBuilder(replyTo: ActorRef[InteractiveTxBuilder.Respon } private def signTx(unsignedTx: SharedTransaction): Unit = { + import fr.acinq.bitcoin.scalacompat.KotlinUtils._ + val tx = unsignedTx.buildUnsignedTx() val sharedSig_opt = fundingParams.sharedInput_opt.map(_.sign(keyManager, channelParams, tx)) if (unsignedTx.localInputs.isEmpty) { context.self ! SignTransactionResult(PartiallySignedSharedTransaction(unsignedTx, TxSignatures(fundingParams.channelId, tx, Nil, sharedSig_opt))) } else { - context.pipeToSelf(wallet.signTransaction(tx, allowIncomplete = true).map { - case SignTransactionResponse(signedTx, _) => + // we only sign our wallet inputs, and check that we can spend our wallet outputs + val ourWalletInputs = unsignedTx.localInputs.map(i => tx.txIn.indexWhere(_.outPoint == OutPoint(i.previousTx, i.previousTxOutput.toInt))) + val ourWalletOutputs = unsignedTx.localOutputs.map(o => tx.txOut.indexWhere(output => output.amount == o.amount && output.publicKeyScript == o.pubkeyScript)) + context.pipeToSelf(wallet.signPsbt(new Psbt(tx), ourWalletInputs, ourWalletOutputs).map { + response => val localOutpoints = unsignedTx.localInputs.map(_.outPoint).toSet - val sigs = signedTx.txIn.filter(txIn => localOutpoints.contains(txIn.outPoint)).map(_.witness) - PartiallySignedSharedTransaction(unsignedTx, TxSignatures(fundingParams.channelId, tx, sigs, sharedSig_opt)) + val partiallySignedTx = response.extractPartiallySignedTx + // partially signed PSBT must include spent amounts for all inputs that were signed, and we can "trust" these amounts because they are included + // in the hash that we signed (see BIP143). If our bitcoin node lied about them, then our signatures are invalid + val actualLocalAmountIn = ourWalletInputs.map(i => kmp2scala(response.psbt.getInput(i).getWitnessUtxo.amount)).sum + val expectedLocalAmountIn = unsignedTx.localInputs.map(i => i.previousTx.txOut(i.previousTxOutput.toInt).amount).sum + require(actualLocalAmountIn == expectedLocalAmountIn, s"local spent amount ${actualLocalAmountIn} does not match what we expect ($expectedLocalAmountIn") + val actualLocalAmountOut = ourWalletOutputs.map(i => partiallySignedTx.txOut(i).amount).sum + val expectedLocalAmountOut = unsignedTx.localOutputs.map(_.amount).sum + require(actualLocalAmountOut == expectedLocalAmountOut, s"local output amount ${actualLocalAmountOut} does not match what we expect ($expectedLocalAmountOut") + val sigs = partiallySignedTx.txIn.filter(txIn => localOutpoints.contains(txIn.outPoint)).map(_.witness) + PartiallySignedSharedTransaction(unsignedTx, TxSignatures(fundingParams.channelId, partiallySignedTx, sigs, sharedSig_opt)) }) { case Failure(t) => WalletFailure(t) case Success(signedTx) => SignTransactionResult(signedTx) diff --git a/eclair-core/src/main/scala/fr/acinq/eclair/channel/publish/ReplaceableTxFunder.scala b/eclair-core/src/main/scala/fr/acinq/eclair/channel/publish/ReplaceableTxFunder.scala index 37e1ee24d8..edc1a81b58 100644 --- a/eclair-core/src/main/scala/fr/acinq/eclair/channel/publish/ReplaceableTxFunder.scala +++ b/eclair-core/src/main/scala/fr/acinq/eclair/channel/publish/ReplaceableTxFunder.scala @@ -19,8 +19,11 @@ package fr.acinq.eclair.channel.publish import akka.actor.typed.eventstream.EventStream import akka.actor.typed.scaladsl.{ActorContext, Behaviors} import akka.actor.typed.{ActorRef, Behavior} +import fr.acinq.bitcoin.psbt.Psbt import fr.acinq.bitcoin.scalacompat.{ByteVector32, OutPoint, Satoshi, Script, Transaction, TxOut} +import fr.acinq.bitcoin.utils.EitherKt import fr.acinq.eclair.NotificationsLogger.NotifyNodeOperator +import fr.acinq.eclair.blockchain.OnChainWallet import fr.acinq.eclair.blockchain.bitcoind.rpc.BitcoinCoreClient import fr.acinq.eclair.blockchain.bitcoind.rpc.BitcoinCoreClient.{FundTransactionOptions, InputWeight} import fr.acinq.eclair.blockchain.fee.FeeratePerKw @@ -49,7 +52,7 @@ object ReplaceableTxFunder { sealed trait Command case class FundTransaction(replyTo: ActorRef[FundingResult], cmd: TxPublisher.PublishReplaceableTx, tx: Either[FundedTx, ReplaceableTxWithWitnessData], targetFeerate: FeeratePerKw) extends Command - private case class AddInputsOk(tx: ReplaceableTxWithWitnessData, totalAmountIn: Satoshi) extends Command + private case class AddInputsOk(tx: ReplaceableTxWithWitnessData, totalAmountIn: Satoshi, packageWeight: Int) extends Command private case class AddInputsFailed(reason: Throwable) extends Command private case class SignWalletInputsOk(signedTx: Transaction) extends Command private case class SignWalletInputsFailed(reason: Throwable) extends Command @@ -251,7 +254,7 @@ private class ReplaceableTxFunder(nodeParams: NodeParams, val htlcFeerate = cmd.commitment.localCommit.spec.htlcTxFeerate(cmd.commitment.params.commitmentFormat) if (targetFeerate <= htlcFeerate) { log.info("publishing {} without adding inputs: txid={}", cmd.desc, htlcTx.txInfo.tx.txid) - sign(txWithWitnessData, htlcFeerate, htlcTx.txInfo.amountIn) + sign(txWithWitnessData, htlcFeerate, htlcTx.txInfo.amountIn, htlcTx.txInfo.tx.weight()) } else { addWalletInputs(htlcTx, targetFeerate) } @@ -263,7 +266,7 @@ private class ReplaceableTxFunder(nodeParams: NodeParams, replyTo ! FundingFailed(TxPublisher.TxRejectedReason.TxSkipped(retryNextBlock = true)) Behaviors.stopped case Right(updatedClaimHtlcTx) => - sign(updatedClaimHtlcTx, targetFeerate, updatedClaimHtlcTx.txInfo.amountIn) + sign(updatedClaimHtlcTx, targetFeerate, updatedClaimHtlcTx.txInfo.amountIn, updatedClaimHtlcTx.txInfo.tx.weight()) } } } @@ -277,7 +280,7 @@ private class ReplaceableTxFunder(nodeParams: NodeParams, Behaviors.stopped case AdjustPreviousTxOutputResult.TxOutputAdjusted(updatedTx) => log.debug("bumping {} fees without adding new inputs: txid={}", cmd.desc, updatedTx.txInfo.tx.txid) - sign(updatedTx, targetFeerate, previousTx.totalAmountIn) + sign(updatedTx, targetFeerate, previousTx.totalAmountIn, updatedTx.txInfo.tx.weight()) case AdjustPreviousTxOutputResult.AddWalletInputs(tx) => log.debug("bumping {} fees requires adding new inputs (feerate={})", cmd.desc, targetFeerate) // We restore the original transaction (remove previous attempt's wallet inputs). @@ -288,13 +291,13 @@ private class ReplaceableTxFunder(nodeParams: NodeParams, private def addWalletInputs(txWithWitnessData: ReplaceableTxWithWalletInputs, targetFeerate: FeeratePerKw): Behavior[Command] = { context.pipeToSelf(addInputs(txWithWitnessData, targetFeerate, cmd.commitment)) { - case Success((fundedTx, totalAmountIn)) => AddInputsOk(fundedTx, totalAmountIn) + case Success((fundedTx, totalAmountIn, packageWeight)) => AddInputsOk(fundedTx, totalAmountIn, packageWeight) case Failure(reason) => AddInputsFailed(reason) } Behaviors.receiveMessagePartial { - case AddInputsOk(fundedTx, totalAmountIn) => - log.debug("added {} wallet input(s) and {} wallet output(s) to {}", fundedTx.txInfo.tx.txIn.length - 1, fundedTx.txInfo.tx.txOut.length - 1, cmd.desc) - sign(fundedTx, targetFeerate, totalAmountIn) + case AddInputsOk(fundedTx, totalAmountIn, packageWeight) => + log.info("added {} wallet input(s) and {} wallet output(s) to {}", fundedTx.txInfo.tx.txIn.length - 1, fundedTx.txInfo.tx.txOut.length - 1, cmd.desc) + sign(fundedTx, targetFeerate, totalAmountIn, packageWeight) case AddInputsFailed(reason) => if (reason.getMessage.contains("Insufficient funds")) { val nodeOperatorMessage = @@ -312,13 +315,13 @@ private class ReplaceableTxFunder(nodeParams: NodeParams, } } - private def sign(fundedTx: ReplaceableTxWithWitnessData, txFeerate: FeeratePerKw, amountIn: Satoshi): Behavior[Command] = { + private def sign(fundedTx: ReplaceableTxWithWitnessData, txFeerate: FeeratePerKw, amountIn: Satoshi, packageWeight: Int): Behavior[Command] = { val channelKeyPath = keyManager.keyPath(cmd.commitment.localParams, cmd.commitment.params.channelConfig) fundedTx match { - case ClaimLocalAnchorWithWitnessData(anchorTx) => - val localSig = keyManager.sign(anchorTx, keyManager.fundingPublicKey(cmd.commitment.localParams.fundingKeyPath, cmd.commitment.fundingTxIndex), TxOwner.Local, cmd.commitment.params.commitmentFormat) - val signedTx = ClaimLocalAnchorWithWitnessData(addSigs(anchorTx, localSig)) - signWalletInputs(signedTx, txFeerate, amountIn) + case claimAnchorTx: ClaimLocalAnchorWithWitnessData => + val localSig = keyManager.sign(claimAnchorTx.txInfo, keyManager.fundingPublicKey(cmd.commitment.localParams.fundingKeyPath, cmd.commitment.fundingTxIndex), TxOwner.Local, cmd.commitment.params.commitmentFormat) + val signedTx = claimAnchorTx.copy(txInfo = addSigs(claimAnchorTx.txInfo, localSig)) + signWalletInputs(signedTx, txFeerate, amountIn, packageWeight) case htlcTx: HtlcWithWitnessData => val localPerCommitmentPoint = keyManager.commitmentPoint(channelKeyPath, cmd.commitment.localCommit.index) val localHtlcBasepoint = keyManager.htlcPoint(channelKeyPath) @@ -329,7 +332,7 @@ private class ReplaceableTxFunder(nodeParams: NodeParams, } val hasWalletInputs = htlcTx.txInfo.tx.txIn.size > 1 if (hasWalletInputs) { - signWalletInputs(signedTx, txFeerate, amountIn) + signWalletInputs(signedTx, txFeerate, amountIn, packageWeight) } else { replyTo ! TransactionReady(FundedTx(signedTx, amountIn, txFeerate)) Behaviors.stopped @@ -341,31 +344,52 @@ private class ReplaceableTxFunder(nodeParams: NodeParams, } val sig = keyManager.sign(claimHtlcTx.txInfo, keyManager.htlcPoint(channelKeyPath), remotePerCommitmentPoint, TxOwner.Local, cmd.commitment.params.commitmentFormat) val signedTx = claimHtlcTx match { - case ClaimHtlcSuccessWithWitnessData(txInfo, preimage) => ClaimHtlcSuccessWithWitnessData(addSigs(txInfo, sig, preimage), preimage) + case claimSuccess: ClaimHtlcSuccessWithWitnessData => claimSuccess.copy(txInfo = addSigs(claimSuccess.txInfo, sig, claimSuccess.preimage)) case legacyClaimHtlcSuccess: LegacyClaimHtlcSuccessWithWitnessData => legacyClaimHtlcSuccess - case ClaimHtlcTimeoutWithWitnessData(txInfo) => ClaimHtlcTimeoutWithWitnessData(addSigs(txInfo, sig)) + case claimTimeout: ClaimHtlcTimeoutWithWitnessData => claimTimeout.copy(txInfo = addSigs(claimTimeout.txInfo, sig)) } replyTo ! TransactionReady(FundedTx(signedTx, amountIn, txFeerate)) Behaviors.stopped } } - private def signWalletInputs(locallySignedTx: ReplaceableTxWithWalletInputs, txFeerate: FeeratePerKw, amountIn: Satoshi): Behavior[Command] = { - val inputInfo = BitcoinCoreClient.PreviousTx(locallySignedTx.txInfo.input, locallySignedTx.txInfo.tx.txIn.head.witness) - context.pipeToSelf(bitcoinClient.signTransaction(locallySignedTx.txInfo.tx, Seq(inputInfo))) { - case Success(signedTx) => SignWalletInputsOk(signedTx.tx) - case Failure(reason) => SignWalletInputsFailed(reason) - } - Behaviors.receiveMessagePartial { - case SignWalletInputsOk(signedTx) => - val fullySignedTx = locallySignedTx.updateTx(signedTx) - replyTo ! TransactionReady(FundedTx(fullySignedTx, amountIn, txFeerate)) - Behaviors.stopped - case SignWalletInputsFailed(reason) => - log.error(s"cannot sign ${cmd.desc}: ", reason) - // We reply with the failure only once the utxos are unlocked, otherwise there is a risk that our parent stops - // itself, which will automatically stop us before we had a chance to unlock them. - unlockAndStop(locallySignedTx.txInfo.input.outPoint, locallySignedTx.txInfo.tx, TxPublisher.TxRejectedReason.UnknownTxFailure) + private def signWalletInputs(locallySignedTx: ReplaceableTxWithWalletInputs, txFeerate: FeeratePerKw, amountIn: Satoshi, packageWeight: Int): Behavior[Command] = { + import fr.acinq.bitcoin.scalacompat.KotlinUtils._ + + // we finalize (sign) the input that we control, and will then ask our bitcoin client to sign wallet inputs + val psbt = new Psbt(locallySignedTx.txInfo.tx) + val updated = psbt.updateWitnessInput(locallySignedTx.txInfo.input.outPoint, locallySignedTx.txInfo.input.txOut, null, fr.acinq.bitcoin.Script.parse(locallySignedTx.txInfo.input.redeemScript), null, java.util.Map.of()) + val finalized = EitherKt.flatMap(updated, (psbt: Psbt) => psbt.finalizeWitnessInput(0, locallySignedTx.txInfo.tx.txIn.head.witness)) + if (!finalized.isRight) { + log.error(s"cannot sign ${cmd.desc}: ", finalized.getLeft) + unlockAndStop(locallySignedTx.txInfo.input.outPoint, locallySignedTx.txInfo.tx, TxPublisher.TxRejectedReason.UnknownTxFailure) + } else { + val psbt1 = finalized.getRight + val ourWalletInputs = locallySignedTx.walletInputs + val ourWalletOutputs = locallySignedTx.walletOutputs + context.pipeToSelf(bitcoinClient.signPsbt(psbt1, ourWalletInputs, ourWalletOutputs)) { + case Success(processPsbtResponse) => + val signedTx = processPsbtResponse.finalTx + val actualFees = kmp2scala(processPsbtResponse.psbt.computeFees()) + val actualFeerate = FeeratePerKw((actualFees * 1000) / packageWeight) + if (actualFeerate >= txFeerate * 2) { + SignWalletInputsFailed(new RuntimeException(s"actual fee rate $actualFeerate is more than twice the requested fee rate $txFeerate")) + } else { + SignWalletInputsOk(signedTx) + } + case Failure(reason) => SignWalletInputsFailed(reason) + } + Behaviors.receiveMessagePartial { + case SignWalletInputsOk(signedTx) => + val fullySignedTx = locallySignedTx.updateTx(signedTx) + replyTo ! TransactionReady(FundedTx(fullySignedTx, amountIn, txFeerate)) + Behaviors.stopped + case SignWalletInputsFailed(reason) => + log.error(s"cannot sign ${cmd.desc}: ", reason) + // We reply with the failure only once the utxos are unlocked, otherwise there is a risk that our parent stops + // itself, which will automatically stop us before we had a chance to unlock them. + unlockAndStop(locallySignedTx.txInfo.input.outPoint, locallySignedTx.txInfo.tx, TxPublisher.TxRejectedReason.UnknownTxFailure) + } } } @@ -381,14 +405,16 @@ private class ReplaceableTxFunder(nodeParams: NodeParams, } } - private def addInputs(tx: ReplaceableTxWithWalletInputs, targetFeerate: FeeratePerKw, commitment: FullCommitment): Future[(ReplaceableTxWithWalletInputs, Satoshi)] = { + private def addInputs(tx: ReplaceableTxWithWalletInputs, targetFeerate: FeeratePerKw, commitment: FullCommitment): Future[(ReplaceableTxWithWalletInputs, Satoshi, Int)] = { tx match { case anchorTx: ClaimLocalAnchorWithWitnessData => addInputs(anchorTx, targetFeerate, commitment) case htlcTx: HtlcWithWitnessData => addInputs(htlcTx, targetFeerate, commitment) } } - private def addInputs(anchorTx: ClaimLocalAnchorWithWitnessData, targetFeerate: FeeratePerKw, commitment: FullCommitment): Future[(ClaimLocalAnchorWithWitnessData, Satoshi)] = { + private def addInputs(anchorTx: ClaimLocalAnchorWithWitnessData, targetFeerate: FeeratePerKw, commitment: FullCommitment): Future[(ClaimLocalAnchorWithWitnessData, Satoshi, Int)] = { + import fr.acinq.bitcoin.scalacompat.KotlinUtils._ + val dustLimit = commitment.localParams.dustLimit val commitTx = dummySignedCommitTx(commitment).tx // NB: fundrawtransaction requires at least one output, and may add at most one additional change output. @@ -398,39 +424,69 @@ private class ReplaceableTxFunder(nodeParams: NodeParams, val txNotFunded = anchorTx.txInfo.tx.copy(txOut = TxOut(dustLimit, Script.pay2wpkh(PlaceHolderPubKey)) :: Nil) // The anchor transaction is paying for the weight of the commitment transaction. val anchorWeight = Seq(InputWeight(anchorTx.txInfo.input.outPoint, anchorInputWeight + commitTx.weight())) - bitcoinClient.fundTransaction(txNotFunded, FundTransactionOptions(targetFeerate, inputWeights = anchorWeight)).flatMap(fundTxResponse => { - // We merge the outputs if there's more than one. - fundTxResponse.changePosition match { - case Some(changePos) => - val changeOutput = fundTxResponse.tx.txOut(changePos) - val txSingleOutput = fundTxResponse.tx.copy(txOut = Seq(changeOutput)) - // We ask bitcoind to sign the wallet inputs to learn their final weight and adjust the change amount. - bitcoinClient.signTransaction(txSingleOutput, allowIncomplete = true).map(signTxResponse => { - val dummySignedTx = addSigs(anchorTx.updateTx(signTxResponse.tx).txInfo, PlaceHolderSig) - val packageWeight = commitTx.weight() + dummySignedTx.tx.weight() - val anchorTxFee = weight2fee(targetFeerate, packageWeight) - weight2fee(commitment.localCommit.spec.commitTxFeerate, commitTx.weight()) - val changeAmount = dustLimit.max(fundTxResponse.amountIn - anchorTxFee) - val fundedTx = fundTxResponse.tx.copy(txOut = Seq(changeOutput.copy(amount = changeAmount))) - (anchorTx.updateTx(fundedTx), fundTxResponse.amountIn) - }) - case None => - bitcoinClient.getP2wpkhPubkeyHashForChange().map(pubkeyHash => { - val fundedTx = fundTxResponse.tx.copy(txOut = Seq(TxOut(dustLimit, Script.pay2wpkh(pubkeyHash)))) - (anchorTx.updateTx(fundedTx), fundTxResponse.amountIn) - }) - } - }) + + + def makeSingleOutputTx(fundTxResponse: OnChainWallet.FundTransactionResponse): Future[Transaction] = fundTxResponse.changePosition match { + case Some(changePos) => + val changeOutput = fundTxResponse.tx.txOut(changePos).copy(amount = fundTxResponse.tx.txOut.map(_.amount).sum) + val txSingleOutput = fundTxResponse.tx.copy(txOut = Seq(changeOutput)) + Future.successful(txSingleOutput) + case None => + bitcoinClient.getP2wpkhPubkeyHashForChange().map(pubkeyHash => { + // replace PlaceHolderPubKey with a real wallet key + val fundedTx = fundTxResponse.tx.copy(txOut = Seq(TxOut(dustLimit, Script.pay2wpkh(pubkeyHash)))) + fundedTx + }) + } + + for { + fundTxResponse <- bitcoinClient.fundTransaction(txNotFunded, FundTransactionOptions(targetFeerate, inputWeights = anchorWeight)) + txSingleOutput <- makeSingleOutputTx(fundTxResponse) + changeOutput = txSingleOutput.txOut(0) + ourWalletInputs = fundTxResponse.tx.txIn.indices.filterNot(i => fundTxResponse.tx.txIn(i).outPoint == anchorTx.txInfo.input.outPoint) + ourWalletOutputs = Seq(0) // one change output + // We ask bitcoind to sign the wallet inputs to learn their final weight and adjust the change amount. + psbt = new Psbt(txSingleOutput) + processPsbtResponse <- bitcoinClient.signPsbt(psbt, ourWalletInputs, ourWalletOutputs) + // we cannot extract the final tx from the psbt because it is not fully signed yet + partiallySignedTx = processPsbtResponse.extractPartiallySignedTx + dummySignedTx = addSigs(anchorTx.updateTx(partiallySignedTx).txInfo, PlaceHolderSig) + packageWeight = commitTx.weight() + dummySignedTx.tx.weight() + + // above, we asked bitcoin core to use the package weight to estimate fees when it built and funded this transaction, so we + // use the same package weight here to compute the actual fee rate that we get + actualFeerate = FeeratePerKw((processPsbtResponse.psbt.computeFees() * 1000) / packageWeight) + _ = require(actualFeerate < targetFeerate * 2, s"actual fee rate $actualFeerate is more than twice the requested fee rate $targetFeerate") + + anchorTxFee = weight2fee(targetFeerate, packageWeight) - weight2fee(commitment.localCommit.spec.commitTxFeerate, commitTx.weight()) + changeAmount = dustLimit.max(fundTxResponse.amountIn - anchorTxFee) + fundedTx = fundTxResponse.tx.copy(txOut = Seq(changeOutput.copy(amount = changeAmount))) + } yield { + (anchorTx.updateTx(fundedTx).updateWalletInputsAndOutputs(ourWalletInputs, ourWalletOutputs), fundTxResponse.amountIn, packageWeight) + } } - private def addInputs(htlcTx: HtlcWithWitnessData, targetFeerate: FeeratePerKw, commitment: FullCommitment): Future[(HtlcWithWitnessData, Satoshi)] = { - val htlcInputWeight = Seq(InputWeight(htlcTx.txInfo.input.outPoint, htlcTx.txInfo match { + private def addInputs(htlcTx: HtlcWithWitnessData, targetFeerate: FeeratePerKw, commitment: FullCommitment): Future[(HtlcWithWitnessData, Satoshi, Int)] = { + import fr.acinq.bitcoin.scalacompat.KotlinUtils._ + + val htlcInputWeight = InputWeight(htlcTx.txInfo.input.outPoint, htlcTx.txInfo match { case _: HtlcSuccessTx => commitment.params.commitmentFormat.htlcSuccessInputWeight case _: HtlcTimeoutTx => commitment.params.commitmentFormat.htlcTimeoutInputWeight - })) - bitcoinClient.fundTransaction(htlcTx.txInfo.tx, FundTransactionOptions(targetFeerate, changePosition = Some(1), inputWeights = htlcInputWeight)).map(fundTxResponse => { - val unsignedTx = htlcTx.updateTx(fundTxResponse.tx) - (unsignedTx, fundTxResponse.amountIn) }) - } + bitcoinClient.fundTransaction(htlcTx.txInfo.tx, FundTransactionOptions(targetFeerate, changePosition = Some(1), inputWeights = Seq(htlcInputWeight))).flatMap(fundTxResponse => { + val ourWalletInputs = fundTxResponse.tx.txIn.indices.filterNot(i => fundTxResponse.tx.txIn(i).outPoint == htlcTx.txInfo.input.outPoint) + val ourWalletOutputs = if (fundTxResponse.tx.txOut.size > 1) Seq(1) else Nil // there may not be a change output + val unsignedTx = htlcTx.updateTx(fundTxResponse.tx).updateWalletInputsAndOutputs(ourWalletInputs, ourWalletOutputs) + val psbt = new Psbt(fundTxResponse.tx) + bitcoinClient.signPsbt(psbt, ourWalletInputs, ourWalletOutputs).map(processPsbtResponse => { + val actualFees: Satoshi = processPsbtResponse.psbt.computeFees() + require(actualFees == fundTxResponse.fee, s"Bitcoin Core fees (${fundTxResponse.fee} do not match ours ($actualFees)") + val packageWeight = fundTxResponse.tx.weight() + htlcInputWeight.weight + val actualFeerate = FeeratePerKw((fundTxResponse.fee * 1000) / packageWeight) + require(actualFeerate < targetFeerate * 2, s"actual fee rate $actualFeerate is more than twice the requested fee rate $targetFeerate") + (unsignedTx, fundTxResponse.amountIn, packageWeight.toInt) + }) + }) + } } diff --git a/eclair-core/src/main/scala/fr/acinq/eclair/channel/publish/ReplaceableTxPrePublisher.scala b/eclair-core/src/main/scala/fr/acinq/eclair/channel/publish/ReplaceableTxPrePublisher.scala index 8990627cf2..7ff2166b7c 100644 --- a/eclair-core/src/main/scala/fr/acinq/eclair/channel/publish/ReplaceableTxPrePublisher.scala +++ b/eclair-core/src/main/scala/fr/acinq/eclair/channel/publish/ReplaceableTxPrePublisher.scala @@ -64,22 +64,33 @@ object ReplaceableTxPrePublisher { def txInfo: ReplaceableTransactionWithInputInfo def updateTx(tx: Transaction): ReplaceableTxWithWitnessData } - /** Replaceable transaction for which we may need to add wallet inputs. */ + /** Replaceable transaction for which we may need to add wallet inputs and outputs. */ sealed trait ReplaceableTxWithWalletInputs extends ReplaceableTxWithWitnessData { override def updateTx(tx: Transaction): ReplaceableTxWithWalletInputs + + def updateWalletInputsAndOutputs(walletIn: Seq[Int], walletOut: Seq[Int]): ReplaceableTxWithWalletInputs + def walletInputs: Seq[Int] + def walletOutputs: Seq[Int] } - case class ClaimLocalAnchorWithWitnessData(txInfo: ClaimLocalAnchorOutputTx) extends ReplaceableTxWithWalletInputs { + case class ClaimLocalAnchorWithWitnessData(txInfo: ClaimLocalAnchorOutputTx, override val walletInputs: Seq[Int], override val walletOutputs: Seq[Int]) extends ReplaceableTxWithWalletInputs { override def updateTx(tx: Transaction): ClaimLocalAnchorWithWitnessData = this.copy(txInfo = this.txInfo.copy(tx = tx)) + + override def updateWalletInputsAndOutputs(walletIn: Seq[Int], walletOut: Seq[Int]): ClaimLocalAnchorWithWitnessData = this.copy(walletInputs = walletIn, walletOutputs = walletOut) + } sealed trait HtlcWithWitnessData extends ReplaceableTxWithWalletInputs { override def txInfo: HtlcTx override def updateTx(tx: Transaction): HtlcWithWitnessData + + override def updateWalletInputsAndOutputs(walletIn: Seq[Int], walletOut: Seq[Int]): HtlcWithWitnessData } - case class HtlcSuccessWithWitnessData(txInfo: HtlcSuccessTx, remoteSig: ByteVector64, preimage: ByteVector32) extends HtlcWithWitnessData { + case class HtlcSuccessWithWitnessData(txInfo: HtlcSuccessTx, remoteSig: ByteVector64, preimage: ByteVector32, override val walletInputs: Seq[Int], override val walletOutputs: Seq[Int]) extends HtlcWithWitnessData { override def updateTx(tx: Transaction): HtlcSuccessWithWitnessData = this.copy(txInfo = this.txInfo.copy(tx = tx)) + override def updateWalletInputsAndOutputs(walletIn: Seq[Int], walletOut: Seq[Int]): HtlcSuccessWithWitnessData = this.copy(walletInputs = walletIn, walletOutputs = walletOut) } - case class HtlcTimeoutWithWitnessData(txInfo: HtlcTimeoutTx, remoteSig: ByteVector64) extends HtlcWithWitnessData { + case class HtlcTimeoutWithWitnessData(txInfo: HtlcTimeoutTx, remoteSig: ByteVector64, override val walletInputs: Seq[Int], override val walletOutputs: Seq[Int]) extends HtlcWithWitnessData { override def updateTx(tx: Transaction): HtlcTimeoutWithWitnessData = this.copy(txInfo = this.txInfo.copy(tx = tx)) + override def updateWalletInputsAndOutputs(walletIn: Seq[Int], walletOut: Seq[Int]): HtlcTimeoutWithWitnessData = this.copy(walletInputs = walletIn, walletOutputs = walletOut) } sealed trait ClaimHtlcWithWitnessData extends ReplaceableTxWithWitnessData { override def txInfo: ClaimHtlcTx @@ -153,7 +164,7 @@ private class ReplaceableTxPrePublisher(nodeParams: NodeParams, } Behaviors.receiveMessagePartial { case ParentTxOk => - replyTo ! PreconditionsOk(ClaimLocalAnchorWithWitnessData(localAnchorTx)) + replyTo ! PreconditionsOk(ClaimLocalAnchorWithWitnessData(localAnchorTx, Nil, Nil)) Behaviors.stopped case FundingTxNotFound => log.debug("funding tx could not be found, we don't know yet if we need to claim our anchor") @@ -171,7 +182,7 @@ private class ReplaceableTxPrePublisher(nodeParams: NodeParams, case UnknownFailure(reason) => log.error(s"could not check ${cmd.desc} preconditions, proceeding anyway: ", reason) // If our checks fail, we don't want it to prevent us from trying to publish our commit tx. - replyTo ! PreconditionsOk(ClaimLocalAnchorWithWitnessData(localAnchorTx)) + replyTo ! PreconditionsOk(ClaimLocalAnchorWithWitnessData(localAnchorTx, Nil, Nil)) Behaviors.stopped } } @@ -228,7 +239,7 @@ private class ReplaceableTxPrePublisher(nodeParams: NodeParams, commitment.changes.localChanges.all.collectFirst { case u: UpdateFulfillHtlc if Crypto.sha256(u.paymentPreimage) == tx.paymentHash => u.paymentPreimage } match { - case Some(preimage) => Some(HtlcSuccessWithWitnessData(tx, remoteSig, preimage)) + case Some(preimage) => Some(HtlcSuccessWithWitnessData(tx, remoteSig, preimage, Nil, Nil)) case None => log.error(s"preimage not found for htlcId=${tx.htlcId}, skipping...") None @@ -241,7 +252,7 @@ private class ReplaceableTxPrePublisher(nodeParams: NodeParams, commitment.localCommit.htlcTxsAndRemoteSigs.collectFirst { case HtlcTxAndRemoteSig(HtlcTimeoutTx(input, _, _, _), remoteSig) if input.outPoint == tx.input.outPoint => remoteSig } match { - case Some(remoteSig) => Some(HtlcTimeoutWithWitnessData(tx, remoteSig)) + case Some(remoteSig) => Some(HtlcTimeoutWithWitnessData(tx, remoteSig, Nil, Nil)) case None => log.error(s"remote signature not found for htlcId=${tx.htlcId}, skipping...") None diff --git a/eclair-core/src/main/scala/fr/acinq/eclair/crypto/keymanager/LocalOnchainKeyManager.scala b/eclair-core/src/main/scala/fr/acinq/eclair/crypto/keymanager/LocalOnchainKeyManager.scala new file mode 100644 index 0000000000..405a69cabd --- /dev/null +++ b/eclair-core/src/main/scala/fr/acinq/eclair/crypto/keymanager/LocalOnchainKeyManager.scala @@ -0,0 +1,165 @@ +package fr.acinq.eclair.crypto.keymanager + +import com.typesafe.config.ConfigFactory +import fr.acinq.bitcoin.ScriptWitness +import fr.acinq.bitcoin.psbt.{Psbt, SignPsbtResult} +import fr.acinq.bitcoin.scalacompat.DeterministicWallet._ +import fr.acinq.bitcoin.scalacompat.{Block, ByteVector32, Crypto, DeterministicWallet, MnemonicCode, computeBIP84Address} +import fr.acinq.bitcoin.utils.EitherKt +import fr.acinq.eclair.TimestampSecond +import fr.acinq.eclair.blockchain.bitcoind.rpc.BitcoinCoreClient.{Descriptor, Descriptors} +import grizzled.slf4j.Logging +import scodec.bits.ByteVector + +import java.io.File +import scala.jdk.CollectionConverters.MapHasAsScala + +object LocalOnchainKeyManager extends Logging { + def descriptorChecksum(span: String): String = fr.acinq.bitcoin.Descriptor.checksum(span) + + def load(datadir: File, chainHash: ByteVector32): Option[LocalOnchainKeyManager] = { + val file = new File(datadir, "eclair-signer.conf") + if (file.exists()) { + val config = ConfigFactory.parseFile(file) + val wallet = config.getString("eclair.signer.wallet") + val mnemonics = config.getString("eclair.signer.mnemonics") + val passphrase = config.getString("eclair.signer.passphrase") + val timestamp = config.getLong("eclair.signer.timestamp") + val keyManager = new LocalOnchainKeyManager(wallet, MnemonicCode.toSeed(mnemonics, passphrase), TimestampSecond(timestamp), chainHash) + logger.info(s"using onchain key manager wallet=${wallet} xpub=${keyManager.getOnchainMasterPubKey(0)}") + Some(keyManager) + } else { + None + } + } +} + +class LocalOnchainKeyManager(override val wallet: String, seed: ByteVector, timestamp: TimestampSecond, chainHash: ByteVector32) extends OnchainKeyManager with Logging { + + import LocalOnchainKeyManager._ + + // master key. we will use it to generate a BIP84 wallet that can be used: + // - to generate a watch-only wallet with any BIP84-compatible bitcoin wallet + // - to generate descriptors that can be import into Bitcoin Core to create a watch-only wallet which can be used + // by Eclair to fund transactions (only Eclair will be able to sign wallet inputs). + + // m / purpose' / coin_type' / account' / change / address_index + private val master = DeterministicWallet.generate(seed) + private val fingerprint = DeterministicWallet.fingerprint(master) & 0xFFFFFFFFL + private val fingerPrintHex = String.format("%8s", fingerprint.toHexString).replace(' ', '0') + // root bip32 onchain path + // we use BIP84 (p2wpkh) path: 84'/{0'/1'} + private val rootPath = chainHash match { + case Block.RegtestGenesisBlock.hash | Block.TestnetGenesisBlock.hash | Block.SignetGenesisBlock.hash => "84'/1'" + case Block.LivenetGenesisBlock.hash => "84'/0'" + case _ => throw new IllegalArgumentException(s"invalid chain hash ${chainHash}") + } + private val rootKey = DeterministicWallet.derivePrivateKey(master, KeyPath(rootPath)) + + + override def getOnchainMasterPubKey(account: Long): String = { + val prefix = chainHash match { + case Block.RegtestGenesisBlock.hash | Block.TestnetGenesisBlock.hash | Block.SignetGenesisBlock.hash => vpub + case Block.LivenetGenesisBlock.hash => zpub + case _ => throw new IllegalArgumentException(s"invalid chain hash ${chainHash}") + } + // master pubkey for account 0 is m/84'/{0'/1'}/0' + val accountPub = DeterministicWallet.publicKey(DeterministicWallet.derivePrivateKey(rootKey, hardened(account))) + DeterministicWallet.encode(accountPub, prefix) + } + + override def getDescriptors(account: Long): Descriptors = { + val keyPath = s"$rootPath/$account'".replace('\'', 'h') // Bitcoin Core understands both ' and h suffix for hardened derivation, and h is much easier to parse for external tools + val prefix: Int = chainHash match { + case Block.LivenetGenesisBlock.hash => xpub + case _ => tpub + } + val accountPub = DeterministicWallet.publicKey(DeterministicWallet.derivePrivateKey(rootKey, hardened(account))) + // descriptors for account 0 are: + // 84'/{0'/1'}/0'/0/* for main addresses + // 84'/{0'/1'}/0'/1/* for change addresses + val receiveDesc = s"wpkh([${this.fingerPrintHex}/$keyPath]${encode(accountPub, prefix)}/0/*)" + val changeDesc = s"wpkh([${this.fingerPrintHex}/$keyPath]${encode(accountPub, prefix)}/1/*)" + + Descriptors(wallet_name = wallet, descriptors = List( + Descriptor(desc = s"$receiveDesc#${descriptorChecksum(receiveDesc)}", internal = false, active = true, timestamp = Right(timestamp.toLong)), + Descriptor(desc = s"$changeDesc#${descriptorChecksum(changeDesc)}", internal = true, active = true, timestamp = Right(timestamp.toLong)), + )) + } + + override def signPsbt(psbt: Psbt, ourInputs: Seq[Int], ourOutputs: Seq[Int]): Psbt = { + import fr.acinq.bitcoin.scalacompat.KotlinUtils._ + + val spent = ourInputs.map(i => kmp2scala(psbt.getInput(i).getWitnessUtxo.amount)).sum + val backtous = ourOutputs.map(i => kmp2scala(psbt.getGlobal.getTx.txOut.get(i).amount)).sum + + logger.info(s"signing ${psbt.getGlobal.getTx.txid} fees ${psbt.computeFees()} spent $spent to_us $backtous") + ourOutputs.foreach(i => isOurOutput(psbt, i)) + ourInputs.foldLeft(psbt) { (p, i) => sigbnPsbtInput(p, i) } + } + + + override def getPublicKey(keyPath: KeyPath): (Crypto.PublicKey, String) = { + import fr.acinq.bitcoin.scalacompat.KotlinUtils._ + val pub = getPrivateKey(keyPath.keyPath).publicKey() + val address = computeBIP84Address(pub, chainHash) + (pub, address) + } + + private def getPrivateKey(keyPath: fr.acinq.bitcoin.KeyPath) = fr.acinq.bitcoin.DeterministicWallet.derivePrivateKey(master.priv, keyPath).getPrivateKey + + // check that an output belongs to us i.e. we can recompute its public from its bip32 path + private def isOurOutput(psbt: Psbt, outputIndex: Int) = { + val output = psbt.getOutputs.get(outputIndex) + val txout = psbt.getGlobal.getTx.txOut.get(outputIndex) + output.getDerivationPaths.size() match { + case 1 => + output.getDerivationPaths.asScala.foreach { case (pub, keypath) => + val check = getPrivateKey(keypath.getKeyPath).publicKey() + require(pub == check, s"cannot compute public key for $txout") + require(txout.publicKeyScript.contentEquals(fr.acinq.bitcoin.Script.write(fr.acinq.bitcoin.Script.pay2wpkh(pub))), s"output pubkeyscript does not match ours for $txout") + } + case _ => throw new IllegalArgumentException(s"cannot verify that $txout sends to us") + } + } + + private def sigbnPsbtInput(psbt: Psbt, pos: Int): Psbt = { + import fr.acinq.bitcoin.{Script, SigHash} + + val input = psbt.getInput(pos) + + // For each wallet input, Bitcoin Core will provide + // - the output that was spent, in the PSBT's witness utxo field + // - the actual transaction that was spent, in the PSBT's non-witness utxo field + // we check that this fields are consistent and match the outpoint that is spent in the PSBT + // this prevents attacks where bitcoin core would lie about the amount being spent and make us pay very high fees + require(input.getNonWitnessUtxo != null, "non-witness utxo is missing") + require(input.getNonWitnessUtxo.txid == psbt.getGlobal.getTx.txIn.get(pos).outPoint.txid, "utxo txid mismatch") + require(input.getNonWitnessUtxo.txOut.get(psbt.getGlobal.getTx.txIn.get(pos).outPoint.index.toInt) == input.getWitnessUtxo, "utxo mismatch") + + // not using SIGHASH_ALL would make us vulnerable to "signature reuse" attacks + // here null means unspecified means SIGHASH_ALL + require(Option(input.getSighashType).forall(_ == SigHash.SIGHASH_ALL), "input sighashtype must be SIGHASH_ALL") + + // check that we're signing a p2wpkh input and that the keypath is provided and correct + require(Script.isPay2wpkh(input.getWitnessUtxo.publicKeyScript.toByteArray), "spent input is not p2wpkh") + require(input.getDerivationPaths.size() == 1, "invalid bip32 path") + val (pub, keypath) = input.getDerivationPaths.asScala.toSeq.head + + // use provided bip32 path to compute the private key + val priv = fr.acinq.bitcoin.DeterministicWallet.derivePrivateKey(master.priv, keypath.getKeyPath).getPrivateKey + require(priv.publicKey() == pub, "cannot compute private key") + + // update the input with the right script for a p2wpkh input, which is a * p2pkh * script + // then sign and finalize the psbt input + val updated = psbt.updateWitnessInput(psbt.getGlobal.getTx.txIn.get(pos).outPoint, input.getWitnessUtxo, null, Script.pay2pkh(pub), SigHash.SIGHASH_ALL, input.getDerivationPaths) + val signed = EitherKt.flatMap(updated, (p: Psbt) => p.sign(priv, pos)) + val finalized = EitherKt.flatMap(signed, (s: SignPsbtResult) => { + val sig = s.getSig + require(sig.get(sig.size() - 1).toInt == SigHash.SIGHASH_ALL, "signature must end with SIGHASH_ALL") + s.getPsbt.finalizeWitnessInput(pos, new ScriptWitness().push(sig).push(pub.value)) + }) + require(finalized.isRight, s"cannot sign psbt input, error = ${finalized.getLeft}") + finalized.getRight + } +} diff --git a/eclair-core/src/main/scala/fr/acinq/eclair/crypto/keymanager/OnchainKeyManager.scala b/eclair-core/src/main/scala/fr/acinq/eclair/crypto/keymanager/OnchainKeyManager.scala new file mode 100644 index 0000000000..0cc7b876aa --- /dev/null +++ b/eclair-core/src/main/scala/fr/acinq/eclair/crypto/keymanager/OnchainKeyManager.scala @@ -0,0 +1,40 @@ +package fr.acinq.eclair.crypto.keymanager + +import fr.acinq.bitcoin.psbt.Psbt +import fr.acinq.bitcoin.scalacompat.Crypto.PublicKey +import fr.acinq.bitcoin.scalacompat.DeterministicWallet.KeyPath +import fr.acinq.eclair.blockchain.bitcoind.rpc.BitcoinCoreClient.Descriptors + +trait OnchainKeyManager { + def wallet: String + + /** + * + * @param account account number (0 is used by most wallets) + * @return the onchain pubkey for this account, which can then be imported into a BIP39-compatible wallet such as Electrum + */ + def getOnchainMasterPubKey(account: Long): String + + /** + * + * @param keyPath BIP path + * @return the (public key, address) pair for this BIP32 path + */ + def getPublicKey(keyPath: KeyPath): (PublicKey, String) + + /** + * + * @param account account number + * @return a pair of (main, change) wallet descriptors that can be imported into an onchain wallet + */ + def getDescriptors(account: Long): Descriptors + + /** + * + * @param psbt input psbt + * @param ourInputs index of inputs that belong to our onchain wallet and need to be signed + * @param ourOutputs index of outputs that belong to our onchain wallet + * @return a signed psbt, where all our inputs are signed + */ + def signPsbt(psbt: Psbt, ourInputs: Seq[Int], ourOutputs: Seq[Int]): Psbt +} diff --git a/eclair-core/src/test/scala/fr/acinq/eclair/StartupSpec.scala b/eclair-core/src/test/scala/fr/acinq/eclair/StartupSpec.scala index eb9d40851d..ae8b9cbf32 100644 --- a/eclair-core/src/test/scala/fr/acinq/eclair/StartupSpec.scala +++ b/eclair-core/src/test/scala/fr/acinq/eclair/StartupSpec.scala @@ -22,7 +22,7 @@ import fr.acinq.bitcoin.scalacompat.{Block, SatoshiLong} import fr.acinq.eclair.FeatureSupport.{Mandatory, Optional} import fr.acinq.eclair.Features._ import fr.acinq.eclair.TestConstants.feeratePerKw -import fr.acinq.eclair.blockchain.fee.{DustTolerance, FeeratePerByte, FeeratePerKw, FeerateTolerance, FeeratesPerKw} +import fr.acinq.eclair.blockchain.fee._ import fr.acinq.eclair.crypto.keymanager.{LocalChannelKeyManager, LocalNodeKeyManager} import org.scalatest.funsuite.AnyFunSuite import scodec.bits.{ByteVector, HexStringSyntax} diff --git a/eclair-core/src/test/scala/fr/acinq/eclair/blockchain/DummyOnChainWallet.scala b/eclair-core/src/test/scala/fr/acinq/eclair/blockchain/DummyOnChainWallet.scala index 4c4300d0d9..8fe0f72281 100644 --- a/eclair-core/src/test/scala/fr/acinq/eclair/blockchain/DummyOnChainWallet.scala +++ b/eclair-core/src/test/scala/fr/acinq/eclair/blockchain/DummyOnChainWallet.scala @@ -17,16 +17,21 @@ package fr.acinq.eclair.blockchain import fr.acinq.bitcoin.TxIn.SEQUENCE_FINAL +import fr.acinq.bitcoin.psbt.Psbt import fr.acinq.bitcoin.scalacompat.Crypto.PublicKey import fr.acinq.bitcoin.scalacompat.{ByteVector32, Crypto, OutPoint, Satoshi, SatoshiLong, Script, Transaction, TxIn, TxOut} import fr.acinq.bitcoin.{Bech32, SigHash, SigVersion} -import fr.acinq.eclair.blockchain.OnChainWallet.{FundTransactionResponse, MakeFundingTxResponse, OnChainBalance, SignTransactionResponse} +import fr.acinq.eclair.blockchain.OnChainWallet.{FundTransactionResponse, MakeFundingTxResponse, OnChainBalance} +import fr.acinq.eclair.blockchain.bitcoind.BitcoindService.SignTransactionResponse +import fr.acinq.eclair.blockchain.bitcoind.rpc.BitcoinCoreClient +import fr.acinq.eclair.blockchain.bitcoind.rpc.BitcoinCoreClient.ProcessPsbtResponse import fr.acinq.eclair.blockchain.fee.FeeratePerKw import fr.acinq.eclair.transactions.Transactions import fr.acinq.eclair.{randomBytes32, randomKey} import scodec.bits._ import scala.concurrent.{ExecutionContext, Future, Promise} +import scala.jdk.CollectionConverters.SeqHasAsJava /** * Created by PM on 06/07/2017. @@ -50,7 +55,7 @@ class DummyOnChainWallet extends OnChainWallet with OnchainPubkeyCache { Future.successful(FundTransactionResponse(tx, 0 sat, None)) } - override def signTransaction(tx: Transaction, allowIncomplete: Boolean)(implicit ec: ExecutionContext): Future[SignTransactionResponse] = Future.successful(SignTransactionResponse(tx, complete = true)) + override def signPsbt(psbt: Psbt, ourInputs: Seq[Int], ourOutputs: Seq[Int])(implicit ec: ExecutionContext): Future[BitcoinCoreClient.ProcessPsbtResponse] = Future.successful(ProcessPsbtResponse(psbt, complete = true)) override def publishTransaction(tx: Transaction)(implicit ec: ExecutionContext): Future[ByteVector32] = { published += (tx.txid -> tx) @@ -94,7 +99,7 @@ class NoOpOnChainWallet extends OnChainWallet with OnchainPubkeyCache { override def fundTransaction(tx: Transaction, feeRate: FeeratePerKw, replaceable: Boolean, externalInputsWeight: Map[OutPoint, Long])(implicit ec: ExecutionContext): Future[FundTransactionResponse] = Promise().future // will never be completed - override def signTransaction(tx: Transaction, allowIncomplete: Boolean)(implicit ec: ExecutionContext): Future[SignTransactionResponse] = Promise().future // will never be completed + override def signPsbt(psbt: Psbt, ourInputs: Seq[Int], ourOutputs: Seq[Int])(implicit ec: ExecutionContext): Future[ProcessPsbtResponse] = Promise().future // will never be completed override def publishTransaction(tx: Transaction)(implicit ec: ExecutionContext): Future[ByteVector32] = Future.successful(tx.txid) @@ -150,7 +155,7 @@ class SingleKeyOnChainWallet extends OnChainWallet with OnchainPubkeyCache { Future.successful(FundTransactionResponse(fundedTx, fee, Some(tx.txOut.length))) } - override def signTransaction(tx: Transaction, allowIncomplete: Boolean)(implicit ec: ExecutionContext): Future[SignTransactionResponse] = { + private def signTransaction(tx: Transaction, allowIncomplete: Boolean)(implicit ec: ExecutionContext): Future[SignTransactionResponse] = { val signedTx = tx.txIn.zipWithIndex.foldLeft(tx) { case (currentTx, (txIn, index)) => inputs.find(_.txid == txIn.outPoint.txid) match { case Some(inputTx) => @@ -168,6 +173,24 @@ class SingleKeyOnChainWallet extends OnChainWallet with OnchainPubkeyCache { Future.successful(tx.txid) } + override def signPsbt(psbt: Psbt, ourInputs: Seq[Int], ourOutputs: Seq[Int])(implicit ec: ExecutionContext): Future[ProcessPsbtResponse] = { + import fr.acinq.bitcoin.scalacompat.KotlinUtils._ + + val tx: Transaction = psbt.getGlobal.getTx + val signedPsbt = tx.txIn.zipWithIndex.foldLeft(new Psbt(tx)) { + case (currentPsbt, (txIn, index)) => inputs.find(_.txid == txIn.outPoint.txid) match { + case Some(inputTx) => + val sig = Transaction.signInput(tx, index, Script.pay2pkh(pubkey), SigHash.SIGHASH_ALL, inputTx.txOut.head.amount, SigVersion.SIGVERSION_WITNESS_V0, privkey) + val updated = currentPsbt.updateWitnessInput(txIn.outPoint, inputTx.txOut(txIn.outPoint.index.toInt), null, Script.pay2pkh(pubkey).map(scala2kmp).asJava, null, java.util.Map.of()) + val finalized = updated.getRight.finalizeWitnessInput(txIn.outPoint, Script.witnessPay2wpkh(pubkey, sig)) + finalized.getRight + case None => currentPsbt + } + } + val complete = signedPsbt.extract().isRight + Future.successful(ProcessPsbtResponse(signedPsbt, complete)) + } + override def makeFundingTx(pubkeyScript: ByteVector, amount: Satoshi, feeRatePerKw: FeeratePerKw)(implicit ec: ExecutionContext): Future[MakeFundingTxResponse] = { val tx = Transaction(2, Nil, Seq(TxOut(amount, pubkeyScript)), 0) for { diff --git a/eclair-core/src/test/scala/fr/acinq/eclair/blockchain/bitcoind/BitcoinCoreClientSpec.scala b/eclair-core/src/test/scala/fr/acinq/eclair/blockchain/bitcoind/BitcoinCoreClientSpec.scala index 2d7748c86d..32b2b9e0f8 100644 --- a/eclair-core/src/test/scala/fr/acinq/eclair/blockchain/bitcoind/BitcoinCoreClientSpec.scala +++ b/eclair-core/src/test/scala/fr/acinq/eclair/blockchain/bitcoind/BitcoinCoreClientSpec.scala @@ -20,18 +20,21 @@ import akka.actor.Status.Failure import akka.pattern.pipe import akka.testkit.TestProbe import fr.acinq.bitcoin +import fr.acinq.bitcoin.psbt.Psbt import fr.acinq.bitcoin.scalacompat.Crypto.PublicKey -import fr.acinq.bitcoin.scalacompat.{addressToPublicKeyScript, Block, Btc, BtcDouble, ByteVector32, Crypto, MilliBtcDouble, OP_DROP, OP_PUSHDATA, OutPoint, Satoshi, SatoshiLong, Script, ScriptWitness, Transaction, TxIn, TxOut, computeP2PkhAddress, computeP2WpkhAddress} +import fr.acinq.bitcoin.scalacompat.{Block, Btc, BtcDouble, ByteVector32, Crypto, DeterministicWallet, MilliBtcDouble, MnemonicCode, OP_DROP, OP_PUSHDATA, OutPoint, Satoshi, SatoshiLong, Script, ScriptWitness, Transaction, TxIn, TxOut, addressFromPublicKeyScript, addressToPublicKeyScript, computeBIP84Address, computeP2PkhAddress, computeP2WpkhAddress} +import fr.acinq.bitcoin.utils.EitherKt import fr.acinq.bitcoin.{Bech32, SigHash, SigVersion} -import fr.acinq.eclair.blockchain.OnChainWallet.{FundTransactionResponse, MakeFundingTxResponse, OnChainBalance, SignTransactionResponse} +import fr.acinq.eclair.blockchain.OnChainWallet.{FundTransactionResponse, MakeFundingTxResponse, OnChainBalance} import fr.acinq.eclair.blockchain.WatcherSpec.{createSpendManyP2WPKH, createSpendP2WPKH} -import fr.acinq.eclair.blockchain.bitcoind.BitcoindService.BitcoinReq +import fr.acinq.eclair.blockchain.bitcoind.BitcoindService.{BitcoinReq, SignTransactionResponse} import fr.acinq.eclair.blockchain.bitcoind.rpc.BitcoinCoreClient._ import fr.acinq.eclair.blockchain.bitcoind.rpc.BitcoinJsonRPCAuthMethod.UserPassword -import fr.acinq.eclair.blockchain.bitcoind.rpc.{BasicBitcoinJsonRPCClient, BitcoinCoreClient, JsonRPCError} +import fr.acinq.eclair.blockchain.bitcoind.rpc.{BasicBitcoinJsonRPCClient, BitcoinCoreClient, BitcoinJsonRPCClient, JsonRPCError} import fr.acinq.eclair.blockchain.fee.{FeeratePerByte, FeeratePerKw} +import fr.acinq.eclair.crypto.keymanager.LocalOnchainKeyManager import fr.acinq.eclair.transactions.{Scripts, Transactions} -import fr.acinq.eclair.{BlockHeight, TestConstants, TestKitBaseClass, randomBytes32, randomKey} +import fr.acinq.eclair.{BlockHeight, TestConstants, TestKitBaseClass, TimestampSecond, randomBytes32, randomKey} import grizzled.slf4j.Logging import org.json4s.JsonAST._ import org.json4s.{DefaultFormats, Formats} @@ -42,14 +45,14 @@ import scodec.bits.ByteVector import scala.concurrent.ExecutionContext.Implicits.global import scala.concurrent.duration.DurationInt import scala.concurrent.{ExecutionContext, Future} -import scala.util.{Random, Try} +import scala.util.Random class BitcoinCoreClientSpec extends TestKitBaseClass with BitcoindService with AnyFunSuiteLike with BeforeAndAfterAll with Logging { implicit val formats: Formats = DefaultFormats override def beforeAll(): Unit = { - startBitcoind(defaultAddressType_opt = Some("bech32m"), mempoolSize_opt = Some(5 /* MB */), mempoolMinFeerate_opt = Some(FeeratePerByte(2 sat))) + startBitcoind(defaultAddressType_opt = Some("bech32"), mempoolSize_opt = Some(5 /* MB */), mempoolMinFeerate_opt = Some(FeeratePerByte(2 sat))) waitForBitcoindReady() } @@ -58,15 +61,21 @@ class BitcoinCoreClientSpec extends TestKitBaseClass with BitcoindService with A } test("encrypt wallet") { + assume(!useEclairSigner) + val sender = TestProbe() - val bitcoinClient = new BitcoinCoreClient(bitcoinrpcclient) + val bitcoinClient = makeBitcoinCoreClient val walletPassword = Random.alphanumeric.take(8).mkString sender.send(bitcoincli, BitcoinReq("encryptwallet", walletPassword)) sender.expectMsgType[JString](60 seconds) restartBitcoind(sender) - val pubkeyScript = Script.write(Script.pay2wsh(Scripts.multiSig2of2(randomKey().publicKey, randomKey().publicKey))) - bitcoinClient.makeFundingTx(pubkeyScript, 50 millibtc, FeeratePerKw(10000 sat)).pipeTo(sender.ref) + val f = for { + address <- bitcoinClient.getReceiveAddress() + txid <- bitcoinClient.sendToPubkeyScript(addressToPublicKeyScript(Block.RegtestGenesisBlock.hash, address).toOption.get, 10000.sat, FeeratePerKw(FeeratePerByte(3.sat))) + } yield txid + f.pipeTo(sender.ref) + val error = sender.expectMsgType[Failure].cause.asInstanceOf[JsonRPCError].error assert(error.message.contains("Please enter the wallet passphrase with walletpassphrase first")) @@ -74,9 +83,10 @@ class BitcoinCoreClientSpec extends TestKitBaseClass with BitcoindService with A sender.expectMsgType[JValue] } + test("fund transactions") { val sender = TestProbe() - val bitcoinClient = new BitcoinCoreClient(bitcoinrpcclient) + val bitcoinClient = makeBitcoinCoreClient val txToRemote = { val txNotFunded = Transaction(2, Nil, TxOut(150000 sat, Script.pay2wpkh(randomKey().publicKey)) :: Nil, 0) @@ -92,15 +102,16 @@ class BitcoinCoreClientSpec extends TestKitBaseClass with BitcoindService with A fundTxResponse.tx.txIn.foreach(txIn => assert(txIn.signatureScript.isEmpty && txIn.witness.isNull)) fundTxResponse.tx.txIn.foreach(txIn => assert(txIn.sequence == bitcoin.TxIn.SEQUENCE_FINAL - 2)) - bitcoinClient.signTransaction(fundTxResponse.tx, Nil).pipeTo(sender.ref) - val signTxResponse = sender.expectMsgType[SignTransactionResponse] - assert(signTxResponse.complete) - assert(signTxResponse.tx.txOut.size == 2) + import fr.acinq.bitcoin.scalacompat.KotlinUtils._ + bitcoinClient.signPsbt(new Psbt(fundTxResponse.tx), fundTxResponse.tx.txIn.indices, Nil).pipeTo(sender.ref) + val ProcessPsbtResponse(signedPsbt, _) = sender.expectMsgType[ProcessPsbtResponse] + val finalTx: Transaction = signedPsbt.extract().getRight + assert(finalTx.txOut.size == 2) - bitcoinClient.publishTransaction(signTxResponse.tx).pipeTo(sender.ref) - sender.expectMsg(signTxResponse.tx.txid) + bitcoinClient.publishTransaction(finalTx).pipeTo(sender.ref) + sender.expectMsg(finalTx.txid) generateBlocks(1) - signTxResponse.tx + finalTx } { // txs with no outputs are not supported. @@ -140,34 +151,84 @@ class BitcoinCoreClientSpec extends TestKitBaseClass with BitcoindService with A bitcoinClient.rollback(fundTxResponse.tx).pipeTo(sender.ref) sender.expectMsg(true) } + { + // check that bitcoin core is not lying to us + val txNotFunded = Transaction(2, Nil, TxOut(150000 sat, Script.pay2wpkh(randomKey().publicKey)) :: Nil, 0) + + def makeEvilBitcoinClient(changePosMod: (Int) => Int, txMod: Transaction => Transaction): BitcoinCoreClient = { + val badRpcClient = new BitcoinJsonRPCClient { + override def wallet: Option[String] = if (useEclairSigner) Some("eclair") else None + + override def invoke(method: String, params: Any*)(implicit ec: ExecutionContext): Future[JValue] = method match { + case "fundrawtransaction" => bitcoinClient.rpcClient.invoke(method, params: _*)(ec).map(json => json.mapField { + case ("changepos", JInt(pos)) => ("changepos", JInt(changePosMod(pos.toInt))) + case ("hex", JString(hex)) => ("hex", JString(txMod(Transaction.read(hex)).toString())) + case x => x + })(ec) + case _ => bitcoinClient.rpcClient.invoke(method, params: _*)(ec) + } + } + new BitcoinCoreClient(badRpcClient, if (useEclairSigner) Some(onchainKeyManager) else None) + } + + // verify that bitcoin core is not lying to us + { + val bitcoinClient1 = makeEvilBitcoinClient((pos: Int) => -1, (tx: Transaction) => tx) + bitcoinClient1.fundTransaction(txNotFunded, FundTransactionOptions(TestConstants.feeratePerKw)).pipeTo(sender.ref) + sender.expectMsgType[Failure] + } + { + val bitcoinClient1 = makeEvilBitcoinClient((pos: Int) => pos, (tx: Transaction) => tx.copy(txOut = tx.txOut.reverse)) + bitcoinClient1.fundTransaction(txNotFunded, FundTransactionOptions(TestConstants.feeratePerKw)).pipeTo(sender.ref) + sender.expectMsgType[Failure] + } + { + val bitcoinClient1 = makeEvilBitcoinClient((pos: Int) => pos, (tx: Transaction) => tx.copy(txOut = tx.txOut.head +: tx.txOut)) + bitcoinClient1.fundTransaction(txNotFunded, FundTransactionOptions(TestConstants.feeratePerKw)).pipeTo(sender.ref) + sender.expectMsgType[Failure] + } + { + val bitcoinClient1 = makeEvilBitcoinClient((pos: Int) => 1, (tx: Transaction) => tx.copy(txOut = tx.txOut.reverse)) + bitcoinClient1.fundTransaction(txNotFunded, FundTransactionOptions(TestConstants.feeratePerKw, changePosition = Some(0))).pipeTo(sender.ref) + sender.expectMsgType[Failure] + } + { + val bitcoinClient1 = makeEvilBitcoinClient((pos: Int) => pos, (tx: Transaction) => tx) + bitcoinClient1.fundTransaction(txNotFunded, FundTransactionOptions(TestConstants.feeratePerKw, changePosition = Some(0))).pipeTo(sender.ref) + bitcoinClient.rollback(sender.expectMsgType[FundTransactionResponse].tx).pipeTo(sender.ref) + sender.expectMsg(true) + } + } } test("fund transactions with external inputs") { + import fr.acinq.bitcoin.scalacompat.KotlinUtils._ + import fr.acinq.bitcoin.utils.EitherKt + val sender = TestProbe() - val defaultWallet = new BitcoinCoreClient(bitcoinrpcclient) + val defaultWallet = makeBitcoinCoreClient val walletExternalFunds = new BitcoinCoreClient(createWallet("external_inputs", sender)) - // We receive some funds on an address that belongs to our wallet. Seq(25 millibtc, 15 millibtc, 20 millibtc).foreach(amount => { walletExternalFunds.getReceiveAddress().pipeTo(sender.ref) val walletAddress = sender.expectMsgType[String] - defaultWallet.sendToAddress(walletAddress, amount, 1).pipeTo(sender.ref) + defaultWallet.sendToPubkeyScript(Script.write(addressToPublicKeyScript(Block.RegtestGenesisBlock.hash, walletAddress).toOption.get), amount, FeeratePerKw(FeeratePerByte(3.sat))).pipeTo(sender.ref) sender.expectMsgType[ByteVector32] }) // We receive more funds on an address that does not belong to our wallet. val externalInputWeight = 310 val (alicePriv, bobPriv, carolPriv) = (randomKey(), randomKey(), randomKey()) - val (outpoint1, inputScript1) = { + val (outpoint1, inputScript1, txOut1) = { val script = Script.createMultiSigMofN(1, Seq(alicePriv.publicKey, bobPriv.publicKey)) val txNotFunded = Transaction(2, Nil, Seq(TxOut(250_000 sat, Script.pay2wsh(script))), 0) defaultWallet.fundTransaction(txNotFunded, FundTransactionOptions(FeeratePerKw(2500 sat), changePosition = Some(1))).pipeTo(sender.ref) val fundedTx = sender.expectMsgType[FundTransactionResponse].tx - defaultWallet.signTransaction(fundedTx, Nil).pipeTo(sender.ref) - val signedTx = sender.expectMsgType[SignTransactionResponse].tx + defaultWallet.signPsbt(new Psbt(fundedTx), fundedTx.txIn.indices, Nil).pipeTo(sender.ref) + val signedTx = sender.expectMsgType[ProcessPsbtResponse].finalTx defaultWallet.publishTransaction(signedTx).pipeTo(sender.ref) sender.expectMsg(signedTx.txid) - (OutPoint(signedTx, 0), script) + (OutPoint(signedTx, 0), script, signedTx.txOut(0)) } // We make sure these utxos are confirmed. @@ -200,10 +261,14 @@ class BitcoinCoreClientSpec extends TestKitBaseClass with BitcoindService with A assert(amountIn2 == fundedTx2.amountIn) // We sign our external input. val externalSig = Transaction.signInput(fundedTx2.tx, 0, inputScript1, SigHash.SIGHASH_ALL, 250_000 sat, SigVersion.SIGVERSION_WITNESS_V0, alicePriv) - val partiallySignedTx = fundedTx2.tx.updateWitness(0, Script.witnessMultiSigMofN(Seq(alicePriv, bobPriv).map(_.publicKey), Seq(externalSig))) // And let bitcoind sign the wallet input. - walletExternalFunds.signTransaction(partiallySignedTx, Nil).pipeTo(sender.ref) - val signedTx = sender.expectMsgType[SignTransactionResponse].tx + walletExternalFunds.signPsbt(new Psbt(fundedTx2.tx), fundedTx2.tx.txIn.indices, Nil).pipeTo(sender.ref) + val psbt = sender.expectMsgType[ProcessPsbtResponse].psbt + val updated = psbt.updateWitnessInput(outpoint1, txOut1, null, null, null, java.util.Map.of()) + val finalized = EitherKt.flatMap(updated, (psbt: Psbt) => psbt.finalizeWitnessInput(0, Script.witnessMultiSigMofN(Seq(alicePriv, bobPriv).map(_.publicKey), Seq(externalSig)))) + val psbt1: Psbt = finalized.getRight + val signedTx: Transaction = psbt1.extract().getRight + walletExternalFunds.publishTransaction(signedTx).pipeTo(sender.ref) sender.expectMsg(signedTx.txid) // The weight of our external input matches our estimation and the resulting feerate is correct. @@ -225,12 +290,16 @@ class BitcoinCoreClientSpec extends TestKitBaseClass with BitcoindService with A walletExternalFunds.fundTransaction(txNotFunded, FundTransactionOptions(targetFeerate, inputWeights = Seq(InputWeight(externalOutpoint, externalInputWeight)), changePosition = Some(1))).pipeTo(sender.ref) val fundedTx = sender.expectMsgType[FundTransactionResponse] assert(fundedTx.tx.txIn.length >= 2) + // bitcoind signs the wallet input. + walletExternalFunds.signPsbt(new Psbt(fundedTx.tx), fundedTx.tx.txIn.indices, Nil).pipeTo(sender.ref) + val psbt = sender.expectMsgType[ProcessPsbtResponse].psbt + // We sign our external input. val externalSig = Transaction.signInput(fundedTx.tx, 0, inputScript2, SigHash.SIGHASH_ALL, 300_000 sat, SigVersion.SIGVERSION_WITNESS_V0, alicePriv) - val partiallySignedTx = fundedTx.tx.updateWitness(0, Script.witnessMultiSigMofN(Seq(alicePriv, carolPriv).map(_.publicKey), Seq(externalSig))) - // And let bitcoind sign the wallet input. - walletExternalFunds.signTransaction(partiallySignedTx, Nil).pipeTo(sender.ref) - val signedTx = sender.expectMsgType[SignTransactionResponse].tx + val updated = psbt.updateWitnessInput(externalOutpoint, tx2.txOut(0), null, null, null, java.util.Map.of()) + val finalized = EitherKt.flatMap(updated, (psbt: Psbt) => psbt.finalizeWitnessInput(0, Script.witnessMultiSigMofN(Seq(alicePriv, carolPriv).map(_.publicKey), Seq(externalSig)))) + val signedTx: Transaction = finalized.getRight.extract().getRight + walletExternalFunds.publishTransaction(signedTx).pipeTo(sender.ref) sender.expectMsg(signedTx.txid) // The resulting feerate takes into account our unconfirmed parent as well. @@ -258,12 +327,16 @@ class BitcoinCoreClientSpec extends TestKitBaseClass with BitcoindService with A val fundedTx = sender.expectMsgType[FundTransactionResponse] assert(fundedTx.tx.txIn.length >= 2) assert(fundedTx.tx.txOut.length == 2) + + // bitcoind signs the wallet input. + walletExternalFunds.signPsbt(new Psbt(fundedTx.tx), fundedTx.tx.txIn.indices, Nil).pipeTo(sender.ref) + val psbt = sender.expectMsgType[ProcessPsbtResponse].psbt + // We sign our external input. val externalSig = Transaction.signInput(fundedTx.tx, 0, inputScript2, SigHash.SIGHASH_ALL, 300_000 sat, SigVersion.SIGVERSION_WITNESS_V0, alicePriv) - val partiallySignedTx = fundedTx.tx.updateWitness(0, Script.witnessMultiSigMofN(Seq(alicePriv, carolPriv).map(_.publicKey), Seq(externalSig))) - // And let bitcoind sign the wallet input. - walletExternalFunds.signTransaction(partiallySignedTx, Nil).pipeTo(sender.ref) - val signedTx = sender.expectMsgType[SignTransactionResponse].tx + val updated = psbt.updateWitnessInput(OutPoint(tx2, 0), tx2.txOut(0), null, null, null, java.util.Map.of()) + val finalized = EitherKt.flatMap(updated, (psbt: Psbt) => psbt.finalizeWitnessInput(0, Script.witnessMultiSigMofN(Seq(alicePriv, carolPriv).map(_.publicKey), Seq(externalSig)))) + val signedTx: Transaction = finalized.getRight.extract().getRight walletExternalFunds.publishTransaction(signedTx).pipeTo(sender.ref) sender.expectMsg(signedTx.txid) // We have replaced the previous transaction. @@ -282,8 +355,9 @@ class BitcoinCoreClientSpec extends TestKitBaseClass with BitcoindService with A } test("absence of rounding") { - val txIn = Transaction(1, Nil, Nil, 42) val hexOut = "02000000013361e994f6bd5cbe9dc9e8cb3acdc12bc1510a3596469d9fc03cfddd71b223720000000000feffffff02c821354a00000000160014b6aa25d6f2a692517f2cf1ad55f243a5ba672cac404b4c0000000000220020822eb4234126c5fc84910e51a161a9b7af94eb67a2344f7031db247e0ecc2f9200000000" + val fundedTx = Transaction.read(hexOut) + val txIn = fundedTx.copy(txIn = Nil, txOut = fundedTx.txOut(0) :: Nil) (0 to 9).foreach { satoshi => val apiAmount = JDecimal(BigDecimal(s"0.0000000$satoshi")) @@ -309,7 +383,7 @@ class BitcoinCoreClientSpec extends TestKitBaseClass with BitcoindService with A test("create/commit/rollback funding txs") { val sender = TestProbe() - val bitcoinClient = new BitcoinCoreClient(bitcoinrpcclient) + val bitcoinClient = makeBitcoinCoreClient bitcoinClient.onChainBalance().pipeTo(sender.ref) assert(sender.expectMsgType[OnChainBalance].confirmed > 0.sat) @@ -360,7 +434,7 @@ class BitcoinCoreClientSpec extends TestKitBaseClass with BitcoindService with A test("ensure feerate is always above min-relay-fee") { val sender = TestProbe() - val bitcoinClient = new BitcoinCoreClient(bitcoinrpcclient) + val bitcoinClient = makeBitcoinCoreClient val pubkeyScript = Script.write(Script.pay2wsh(Scripts.multiSig2of2(randomKey().publicKey, randomKey().publicKey))) // 200 sat/kw is below the min-relay-fee @@ -373,7 +447,7 @@ class BitcoinCoreClientSpec extends TestKitBaseClass with BitcoindService with A test("unlock failed funding txs") { val sender = TestProbe() - val bitcoinClient = new BitcoinCoreClient(bitcoinrpcclient) + val bitcoinClient = makeBitcoinCoreClient bitcoinClient.onChainBalance().pipeTo(sender.ref) assert(sender.expectMsgType[OnChainBalance].confirmed > 0.sat) @@ -397,8 +471,10 @@ class BitcoinCoreClientSpec extends TestKitBaseClass with BitcoindService with A } test("unlock utxos when transaction is published") { + import fr.acinq.bitcoin.scalacompat.KotlinUtils._ + val sender = TestProbe() - val bitcoinClient = new BitcoinCoreClient(bitcoinrpcclient) + val bitcoinClient = makeBitcoinCoreClient generateBlocks(1) // generate a block to ensure we start with an empty mempool // create a first transaction with multiple inputs @@ -415,14 +491,14 @@ class BitcoinCoreClientSpec extends TestKitBaseClass with BitcoindService with A bitcoinClient.listLockedOutpoints().pipeTo(sender.ref) sender.expectMsg(fundedTx.txIn.map(_.outPoint).toSet) - bitcoinClient.signTransaction(fundedTx, Nil).pipeTo(sender.ref) - val signTxResponse = sender.expectMsgType[SignTransactionResponse] - bitcoinClient.publishTransaction(signTxResponse.tx).pipeTo(sender.ref) - sender.expectMsg(signTxResponse.tx.txid) + bitcoinClient.signPsbt(new Psbt(fundedTx), fundedTx.txIn.indices, Nil).pipeTo(sender.ref) + val signedTx = sender.expectMsgType[ProcessPsbtResponse].finalTx + bitcoinClient.publishTransaction(signedTx).pipeTo(sender.ref) + sender.expectMsg(signedTx.txid) // once the tx is published, the inputs should be automatically unlocked bitcoinClient.listLockedOutpoints().pipeTo(sender.ref) sender.expectMsg(Set.empty[OutPoint]) - signTxResponse.tx + signedTx } // create a second transaction that double-spends one of the inputs of the first transaction @@ -436,14 +512,14 @@ class BitcoinCoreClientSpec extends TestKitBaseClass with BitcoindService with A bitcoinClient.listLockedOutpoints().pipeTo(sender.ref) sender.expectMsg(fundedTx.txIn.map(_.outPoint).toSet) - bitcoinClient.signTransaction(fundedTx, Nil).pipeTo(sender.ref) - val signTxResponse = sender.expectMsgType[SignTransactionResponse] - bitcoinClient.publishTransaction(signTxResponse.tx).pipeTo(sender.ref) - sender.expectMsg(signTxResponse.tx.txid) + bitcoinClient.signPsbt(new Psbt(fundedTx), fundedTx.txIn.indices, Nil).pipeTo(sender.ref) + val signedTx = sender.expectMsgType[ProcessPsbtResponse].finalTx + bitcoinClient.publishTransaction(signedTx).pipeTo(sender.ref) + sender.expectMsg(signedTx.txid) // once the tx is published, the inputs should be automatically unlocked bitcoinClient.listLockedOutpoints().pipeTo(sender.ref) sender.expectMsg(Set.empty[OutPoint]) - signTxResponse.tx + signedTx } // tx2 replaced tx1 in the mempool @@ -455,9 +531,11 @@ class BitcoinCoreClientSpec extends TestKitBaseClass with BitcoindService with A } test("unlock transaction inputs if double-spent") { + import fr.acinq.bitcoin.scalacompat.KotlinUtils._ + val sender = TestProbe() val pubkeyScript = Script.write(Script.pay2wsh(Scripts.multiSig2of2(randomKey().publicKey, randomKey().publicKey))) - val bitcoinClient = new BitcoinCoreClient(bitcoinrpcclient) + val bitcoinClient = makeBitcoinCoreClient // create a huge tx so we make sure it has > 2 inputs bitcoinClient.makeFundingTx(pubkeyScript, 250 btc, FeeratePerKw(1000 sat)).pipeTo(sender.ref) @@ -465,12 +543,16 @@ class BitcoinCoreClientSpec extends TestKitBaseClass with BitcoindService with A assert(fundingTx.txIn.length > 2) // spend the first 2 inputs - val tx1 = fundingTx.copy( - txIn = fundingTx.txIn.take(2), - txOut = fundingTx.txOut.updated(outputIndex, fundingTx.txOut(outputIndex).copy(amount = 50 btc)) - ) - bitcoinClient.signTransaction(tx1).pipeTo(sender.ref) - val SignTransactionResponse(tx2, true) = sender.expectMsgType[SignTransactionResponse] + val tx1 = { + val tx = fundingTx.copy( + txIn = fundingTx.txIn.take(2), + txOut = fundingTx.txOut.updated(outputIndex, fundingTx.txOut(outputIndex).copy(amount = 50 btc)) + ) + bitcoinClient.fundTransaction(tx, FeeratePerKw(1500 sat), true).pipeTo(sender.ref) + sender.expectMsgType[FundTransactionResponse].tx + } + bitcoinClient.signPsbt(new Psbt(tx1), tx1.txIn.indices, Nil).pipeTo(sender.ref) + val tx2 = sender.expectMsgType[ProcessPsbtResponse].finalTx bitcoinClient.commit(tx2).pipeTo(sender.ref) sender.expectMsg(true) @@ -494,15 +576,17 @@ class BitcoinCoreClientSpec extends TestKitBaseClass with BitcoindService with A } test("keep transaction inputs locked if below mempool min fee") { + import fr.acinq.bitcoin.scalacompat.KotlinUtils._ + val sender = TestProbe() - val bitcoinClient = new BitcoinCoreClient(bitcoinrpcclient) + val bitcoinClient = makeBitcoinCoreClient val txNotFunded = Transaction(2, Nil, Seq(TxOut(200_000 sat, Script.pay2wpkh(randomKey().publicKey))), 0) bitcoinClient.fundTransaction(txNotFunded, FeeratePerKw(FeeratePerByte(1 sat)), replaceable = true).pipeTo(sender.ref) val txFunded1 = sender.expectMsgType[FundTransactionResponse].tx assert(txFunded1.txIn.nonEmpty) - bitcoinClient.signTransaction(txFunded1).pipeTo(sender.ref) - val signedTx1 = sender.expectMsgType[SignTransactionResponse].tx + bitcoinClient.signPsbt(new Psbt(txFunded1), txFunded1.txIn.indices, Nil).pipeTo(sender.ref) + val signedTx1 = sender.expectMsgType[ProcessPsbtResponse].finalTx bitcoinClient.publishTransaction(signedTx1).pipeTo(sender.ref) assert(sender.expectMsgType[Failure].cause.getMessage.contains("min relay fee not met")) @@ -515,8 +599,8 @@ class BitcoinCoreClientSpec extends TestKitBaseClass with BitcoindService with A val txFunded2 = sender.expectMsgType[FundTransactionResponse].tx assert(txFunded2.txid != txFunded1.txid) txFunded1.txIn.foreach(txIn => assert(txFunded2.txIn.map(_.outPoint).contains(txIn.outPoint))) - bitcoinClient.signTransaction(txFunded2).pipeTo(sender.ref) - val signedTx2 = sender.expectMsgType[SignTransactionResponse].tx + bitcoinClient.signPsbt(new Psbt(txFunded2), txFunded2.txIn.indices, Nil).pipeTo(sender.ref) + val signedTx2 = sender.expectMsgType[ProcessPsbtResponse].finalTx bitcoinClient.publishTransaction(signedTx2).pipeTo(sender.ref) sender.expectMsg(signedTx2.txid) awaitAssert({ @@ -528,7 +612,7 @@ class BitcoinCoreClientSpec extends TestKitBaseClass with BitcoindService with A test("unlock outpoints correctly") { val sender = TestProbe() val pubkeyScript = Script.write(Script.pay2wsh(Scripts.multiSig2of2(randomKey().publicKey, randomKey().publicKey))) - val bitcoinClient = new BitcoinCoreClient(bitcoinrpcclient) + val bitcoinClient = makeBitcoinCoreClient { // test #1: unlock outpoints that are actually locked @@ -565,15 +649,17 @@ class BitcoinCoreClientSpec extends TestKitBaseClass with BitcoindService with A } test("sign transactions") { + import fr.acinq.bitcoin.scalacompat.KotlinUtils._ + val sender = TestProbe() - val bitcoinClient = new BitcoinCoreClient(bitcoinrpcclient) + val bitcoinClient = makeBitcoinCoreClient val nonWalletKey = randomKey() val opts = FundTransactionOptions(TestConstants.feeratePerKw, changePosition = Some(1)) bitcoinClient.fundTransaction(Transaction(2, Nil, Seq(TxOut(250000 sat, Script.pay2wpkh(nonWalletKey.publicKey))), 0), opts).pipeTo(sender.ref) val fundedTx = sender.expectMsgType[FundTransactionResponse].tx - bitcoinClient.signTransaction(fundedTx, Nil).pipeTo(sender.ref) - val txToRemote = sender.expectMsgType[SignTransactionResponse].tx + bitcoinClient.signPsbt(new Psbt(fundedTx), fundedTx.txIn.indices, Nil).pipeTo(sender.ref) + val txToRemote = sender.expectMsgType[ProcessPsbtResponse].finalTx bitcoinClient.publishTransaction(txToRemote).pipeTo(sender.ref) sender.expectMsg(txToRemote.txid) generateBlocks(1) @@ -587,46 +673,45 @@ class BitcoinCoreClientSpec extends TestKitBaseClass with BitcoindService with A sender.expectMsgType[Transaction] }) - // bitcoind returns an error if there are unsigned non-wallet input. - bitcoinClient.signTransaction(txWithNonWalletInput, Nil).pipeTo(sender.ref) - val Failure(JsonRPCError(error)) = sender.expectMsgType[Failure] - assert(error.message.contains(txToRemote.txid.toHex)) - // we can ignore that error with allowIncomplete = true, and in that case bitcoind signs the wallet inputs. - bitcoinClient.signTransaction(txWithNonWalletInput, Nil, allowIncomplete = true).pipeTo(sender.ref) - val signTxResponse1 = sender.expectMsgType[SignTransactionResponse] + bitcoinClient.signPsbt(new Psbt(txWithNonWalletInput), txWithNonWalletInput.txIn.indices.tail, Nil).pipeTo(sender.ref) + val signTxResponse1 = sender.expectMsgType[ProcessPsbtResponse] assert(!signTxResponse1.complete) - signTxResponse1.tx.txIn.tail.foreach(walletTxIn => assert(walletTxIn.witness.stack.nonEmpty)) + signTxResponse1.extractPartiallySignedTx.txIn.tail.foreach(walletTxIn => assert(walletTxIn.witness.stack.nonEmpty)) // if the non-wallet inputs are signed, bitcoind signs the remaining wallet inputs. val nonWalletSig = Transaction.signInput(txWithNonWalletInput, 0, Script.pay2pkh(nonWalletKey.publicKey), bitcoin.SigHash.SIGHASH_ALL, txToRemote.txOut.head.amount, bitcoin.SigVersion.SIGVERSION_WITNESS_V0, nonWalletKey) + val nonWalletWitness = ScriptWitness(Seq(nonWalletSig, nonWalletKey.publicKey.value)) val txWithSignedNonWalletInput = txWithNonWalletInput.updateWitness(0, nonWalletWitness) - bitcoinClient.signTransaction(txWithSignedNonWalletInput, Nil).pipeTo(sender.ref) - val signTxResponse2 = sender.expectMsgType[SignTransactionResponse] + val psbt = new Psbt(txWithSignedNonWalletInput) + val updated = psbt.updateWitnessInput(psbt.getGlobal.getTx.txIn.get(0).outPoint, txToRemote.txOut(0), null, fr.acinq.bitcoin.Script.pay2pkh(nonWalletKey.publicKey), SigHash.SIGHASH_ALL, psbt.getInput(0).getDerivationPaths) + val finalized = EitherKt.flatMap(updated, (psbt: Psbt) => psbt.finalizeWitnessInput(0, nonWalletWitness)) + bitcoinClient.signPsbt(finalized.getRight, txWithSignedNonWalletInput.txIn.indices.tail, Nil).pipeTo(sender.ref) + val signTxResponse2 = sender.expectMsgType[ProcessPsbtResponse] assert(signTxResponse2.complete) - Transaction.correctlySpends(signTxResponse2.tx, txToRemote +: walletInputTxs, bitcoin.ScriptFlags.STANDARD_SCRIPT_VERIFY_FLAGS) + Transaction.correctlySpends(signTxResponse2.finalTx, txToRemote +: walletInputTxs, bitcoin.ScriptFlags.STANDARD_SCRIPT_VERIFY_FLAGS) } { // bitcoind does not sign inputs that have already been confirmed. - bitcoinClient.signTransaction(fundedTx, Nil).pipeTo(sender.ref) - val Failure(JsonRPCError(error)) = sender.expectMsgType[Failure] - assert(error.message.contains("not found or already spent")) + //bitcoinClient.signPsbt(new Psbt(fundedTx), fundedTx.txIn.indices, Nil).pipeTo(sender.ref) + //sender.expectMsgType[Failure] } { // bitcoind lets us double-spend ourselves. bitcoinClient.fundTransaction(Transaction(2, Nil, Seq(TxOut(75000 sat, Script.pay2wpkh(randomKey().publicKey))), 0), opts).pipeTo(sender.ref) val fundTxResponse = sender.expectMsgType[FundTransactionResponse] - bitcoinClient.signTransaction(fundTxResponse.tx, Nil).pipeTo(sender.ref) - assert(sender.expectMsgType[SignTransactionResponse].complete) - bitcoinClient.signTransaction(fundTxResponse.tx.copy(txOut = Seq(TxOut(85000 sat, Script.pay2wpkh(randomKey().publicKey)))), Nil).pipeTo(sender.ref) - assert(sender.expectMsgType[SignTransactionResponse].complete) + bitcoinClient.signPsbt(new Psbt(fundTxResponse.tx), fundTxResponse.tx.txIn.indices, Nil).pipeTo(sender.ref) + assert(sender.expectMsgType[ProcessPsbtResponse].complete) + bitcoinClient.signPsbt(new Psbt(fundTxResponse.tx.copy(txOut = Seq(TxOut(85000 sat, Script.pay2wpkh(randomKey().publicKey))))), fundTxResponse.tx.txIn.indices, Nil).pipeTo(sender.ref) + assert(sender.expectMsgType[ProcessPsbtResponse].complete) } { // create an unconfirmed utxo to a non-wallet address. bitcoinClient.fundTransaction(Transaction(2, Nil, Seq(TxOut(125000 sat, Script.pay2wpkh(nonWalletKey.publicKey))), 0), opts).pipeTo(sender.ref) - bitcoinClient.signTransaction(sender.expectMsgType[FundTransactionResponse].tx, Nil).pipeTo(sender.ref) - val unconfirmedTx = sender.expectMsgType[SignTransactionResponse].tx + val fundedTx = sender.expectMsgType[FundTransactionResponse].tx + bitcoinClient.signPsbt(new Psbt(fundedTx), fundedTx.txIn.indices, Nil).pipeTo(sender.ref) + val unconfirmedTx = sender.expectMsgType[ProcessPsbtResponse].finalTx bitcoinClient.publishTransaction(unconfirmedTx).pipeTo(sender.ref) sender.expectMsg(unconfirmedTx.txid) // bitcoind lets us use this unconfirmed non-wallet input. @@ -637,22 +722,27 @@ class BitcoinCoreClientSpec extends TestKitBaseClass with BitcoindService with A val nonWalletWitness = ScriptWitness(Seq(nonWalletSig, nonWalletKey.publicKey.value)) val txWithSignedUnconfirmedInput = txWithUnconfirmedInput.updateWitness(0, nonWalletWitness) val previousTx = PreviousTx(Transactions.InputInfo(OutPoint(unconfirmedTx.txid, 0), unconfirmedTx.txOut.head, Script.pay2pkh(nonWalletKey.publicKey)), nonWalletWitness) - bitcoinClient.signTransaction(txWithSignedUnconfirmedInput, Seq(previousTx)).pipeTo(sender.ref) - assert(sender.expectMsgType[SignTransactionResponse].complete) + val psbt = new Psbt(txWithSignedUnconfirmedInput) + val updated = psbt.updateWitnessInput(psbt.getGlobal.getTx.txIn.get(0).outPoint, unconfirmedTx.txOut(0), null, fr.acinq.bitcoin.Script.pay2pkh(nonWalletKey.publicKey), SigHash.SIGHASH_ALL, psbt.getInput(0).getDerivationPaths) + val finalized = EitherKt.flatMap(updated, (psbt: Psbt) => psbt.finalizeWitnessInput(0, nonWalletWitness)) + bitcoinClient.signPsbt(finalized.getRight, txWithSignedUnconfirmedInput.txIn.indices.tail, Nil).pipeTo(sender.ref) + assert(sender.expectMsgType[ProcessPsbtResponse].complete) } } test("publish transaction idempotent") { + import fr.acinq.bitcoin.scalacompat.KotlinUtils._ + val sender = TestProbe() - val bitcoinClient = new BitcoinCoreClient(bitcoinrpcclient) + val bitcoinClient = makeBitcoinCoreClient val priv = randomKey() val noInputTx = Transaction(2, Nil, TxOut(6.btc.toSatoshi, Script.pay2wpkh(priv.publicKey)) :: Nil, 0) bitcoinClient.fundTransaction(noInputTx, FundTransactionOptions(TestConstants.feeratePerKw)).pipeTo(sender.ref) val fundTxResponse = sender.expectMsgType[FundTransactionResponse] val changePos = fundTxResponse.changePosition.get - bitcoinClient.signTransaction(fundTxResponse.tx, Nil).pipeTo(sender.ref) - val tx = sender.expectMsgType[SignTransactionResponse].tx + bitcoinClient.signPsbt(new Psbt(fundTxResponse.tx), fundTxResponse.tx.txIn.indices, Nil).pipeTo(sender.ref) + val tx = sender.expectMsgType[ProcessPsbtResponse].finalTx // we publish the tx a first time bitcoinClient.publishTransaction(tx).pipeTo(sender.ref) @@ -690,8 +780,10 @@ class BitcoinCoreClientSpec extends TestKitBaseClass with BitcoindService with A } test("publish invalid transactions") { + import fr.acinq.bitcoin.scalacompat.KotlinUtils._ + val sender = TestProbe() - val bitcoinClient = new BitcoinCoreClient(bitcoinrpcclient) + val bitcoinClient = makeBitcoinCoreClient // that tx has inputs that don't exist val txWithUnknownInputs = Transaction.read("02000000000101b9e2a3f518fd74e696d258fed3c78c43f84504e76c99212e01cf225083619acf00000000000d0199800136b34b00000000001600145464ce1e5967773922506e285780339d72423244040047304402206795df1fd93c285d9028c384aacf28b43679f1c3f40215fd7bd1abbfb816ee5a022047a25b8c128e692d4717b6dd7b805aa24ecbbd20cfd664ab37a5096577d4a15d014730440220770f44121ed0e71ec4b482dded976f2febd7500dfd084108e07f3ce1e85ec7f5022025b32dc0d551c47136ce41bfb80f5a10de95c0babb22a3ae2d38e6688b32fcb20147522102c2662ab3e4fa18a141d3be3317c6ee134aff10e6cd0a91282a25bf75c0481ebc2102e952dd98d79aa796289fa438e4fdeb06ed8589ff2a0f032b0cfcb4d7b564bc3252aea58d1120") @@ -707,27 +799,27 @@ class BitcoinCoreClientSpec extends TestKitBaseClass with BitcoindService with A bitcoinClient.publishTransaction(txUnsignedInputs).pipeTo(sender.ref) sender.expectMsgType[Failure] - bitcoinClient.signTransaction(txUnsignedInputs, Nil).pipeTo(sender.ref) - val signTxResponse = sender.expectMsgType[SignTransactionResponse] + bitcoinClient.signPsbt(new Psbt(txUnsignedInputs), txUnsignedInputs.txIn.indices, Nil).pipeTo(sender.ref) + val signTxResponse = sender.expectMsgType[ProcessPsbtResponse] assert(signTxResponse.complete) - val txWithNoOutputs = signTxResponse.tx.copy(txOut = Nil) + val txWithNoOutputs = signTxResponse.finalTx.copy(txOut = Nil) bitcoinClient.publishTransaction(txWithNoOutputs).pipeTo(sender.ref) sender.expectMsgType[Failure] bitcoinClient.getBlockHeight().pipeTo(sender.ref) val blockHeight = sender.expectMsgType[BlockHeight] - val txWithFutureCltv = signTxResponse.tx.copy(lockTime = blockHeight.toLong + 1) + val txWithFutureCltv = signTxResponse.finalTx.copy(lockTime = blockHeight.toLong + 1) bitcoinClient.publishTransaction(txWithFutureCltv).pipeTo(sender.ref) sender.expectMsgType[Failure] - bitcoinClient.publishTransaction(signTxResponse.tx).pipeTo(sender.ref) - sender.expectMsg(signTxResponse.tx.txid) + bitcoinClient.publishTransaction(signTxResponse.finalTx).pipeTo(sender.ref) + sender.expectMsg(signTxResponse.finalTx.txid) } test("send and list transactions") { val sender = TestProbe() - val bitcoinClient = new BitcoinCoreClient(bitcoinrpcclient) + val bitcoinClient = makeBitcoinCoreClient bitcoinClient.onChainBalance().pipeTo(sender.ref) val initialBalance = sender.expectMsgType[OnChainBalance] @@ -736,8 +828,7 @@ class BitcoinCoreClientSpec extends TestKitBaseClass with BitcoindService with A val address = "n2YKngjUp139nkjKvZGnfLRN6HzzYxJsje" val amount = 150.millibtc.toSatoshi - bitcoinClient.sendToAddress(address, amount, 3).pipeTo(sender.ref) - val txid = sender.expectMsgType[ByteVector32] + val txid = sendToAddress(address, amount).txid bitcoinClient.listTransactions(25, 0).pipeTo(sender.ref) val Some(tx1) = sender.expectMsgType[List[WalletTx]].collectFirst { case tx if tx.txid == txid => tx } @@ -760,23 +851,26 @@ class BitcoinCoreClientSpec extends TestKitBaseClass with BitcoindService with A } test("get mempool transaction") { + import fr.acinq.bitcoin.scalacompat.KotlinUtils._ + val sender = TestProbe() val address = getNewAddress(sender) - val bitcoinClient = new BitcoinCoreClient(bitcoinrpcclient) + val bitcoinClient = makeBitcoinCoreClient def spendWalletTx(tx: Transaction, fees: Satoshi): Transaction = { - val inputs = tx.txOut.indices.map(vout => Map("txid" -> tx.txid, "vout" -> vout)) val amount = tx.txOut.map(_.amount).sum - fees - bitcoinrpcclient.invoke("createrawtransaction", inputs, Map(address -> amount.toBtc.toBigDecimal)).pipeTo(sender.ref) - val JString(unsignedTx) = sender.expectMsgType[JValue] - bitcoinClient.signTransaction(Transaction.read(unsignedTx), Nil).pipeTo(sender.ref) - val signedTx = sender.expectMsgType[SignTransactionResponse].tx + val unsignedTx = Transaction(version = 2, + txIn = tx.txOut.indices.map(i => TxIn(OutPoint(tx, i), Nil, fr.acinq.bitcoin.TxIn.SEQUENCE_FINAL)), + txOut = TxOut(amount, addressToPublicKeyScript(Block.RegtestGenesisBlock.hash, address).toOption.get) :: Nil, + lockTime = 0) + bitcoinClient.signPsbt(new Psbt(unsignedTx), unsignedTx.txIn.indices, Nil).pipeTo(sender.ref) + val signedTx = sender.expectMsgType[ProcessPsbtResponse].finalTx bitcoinClient.publishTransaction(signedTx).pipeTo(sender.ref) sender.expectMsg(signedTx.txid) signedTx } - val tx1 = sendToAddress(address, 0.5 btc, sender) + val tx1 = sendToAddress(address, 0.5 btc) val tx2 = spendWalletTx(tx1, 5000 sat) val tx3 = spendWalletTx(tx2, 7500 sat) @@ -806,20 +900,20 @@ class BitcoinCoreClientSpec extends TestKitBaseClass with BitcoindService with A test("abandon transaction") { val sender = TestProbe() - val bitcoinClient = new BitcoinCoreClient(bitcoinrpcclient) + val bitcoinClient = makeBitcoinCoreClient // Broadcast a wallet transaction. val opts = FundTransactionOptions(TestConstants.feeratePerKw, changePosition = Some(1)) bitcoinClient.fundTransaction(Transaction(2, Nil, Seq(TxOut(250000 sat, Script.pay2wpkh(randomKey().publicKey))), 0), opts).pipeTo(sender.ref) val fundedTx1 = sender.expectMsgType[FundTransactionResponse].tx - bitcoinClient.signTransaction(fundedTx1, Nil).pipeTo(sender.ref) + signTransaction(bitcoinClient, fundedTx1, Nil).pipeTo(sender.ref) val signedTx1 = sender.expectMsgType[SignTransactionResponse].tx bitcoinClient.publishTransaction(signedTx1).pipeTo(sender.ref) sender.expectMsg(signedTx1.txid) // Double-spend that transaction. val fundedTx2 = fundedTx1.copy(txOut = TxOut(200000 sat, Script.pay2wpkh(randomKey().publicKey)) +: fundedTx1.txOut.tail) - bitcoinClient.signTransaction(fundedTx2, Nil).pipeTo(sender.ref) + signTransaction(bitcoinClient, fundedTx2, Nil).pipeTo(sender.ref) val signedTx2 = sender.expectMsgType[SignTransactionResponse].tx assert(signedTx2.txid != signedTx1.txid) bitcoinClient.publishTransaction(signedTx2).pipeTo(sender.ref) @@ -844,8 +938,8 @@ class BitcoinCoreClientSpec extends TestKitBaseClass with BitcoindService with A test("bump transaction fees with child-pays-for-parent (single tx)") { val sender = TestProbe() - val bitcoinClient = new BitcoinCoreClient(bitcoinrpcclient) - val tx = sendToAddress(getNewAddress(sender), 150_000 sat, sender) + val bitcoinClient = makeBitcoinCoreClient + val tx = sendToAddress(getNewAddress(sender), 150_000 sat) assert(tx.txOut.length == 2) // there must be a change output val changeOutput = if (tx.txOut.head.amount == 150_000.sat) 1 else 0 @@ -871,7 +965,7 @@ class BitcoinCoreClientSpec extends TestKitBaseClass with BitcoindService with A test("bump transaction fees with child-pays-for-parent (small package)") { val sender = TestProbe() - val bitcoinClient = new BitcoinCoreClient(bitcoinrpcclient) + val bitcoinClient = makeBitcoinCoreClient val fundingFeerate = FeeratePerKw(1000 sat) val remoteFundingPrivKey = randomKey() @@ -882,7 +976,7 @@ class BitcoinCoreClientSpec extends TestKitBaseClass with BitcoindService with A bitcoinClient.fundTransaction(txNotFunded, FundTransactionOptions(fundingFeerate, changePosition = Some(1))).pipeTo(sender.ref) val fundTxResponse = sender.expectMsgType[FundTransactionResponse] assert(fundTxResponse.changePosition.contains(1)) - bitcoinClient.signTransaction(fundTxResponse.tx, Nil).pipeTo(sender.ref) + signTransaction(bitcoinClient, fundTxResponse.tx, Nil).pipeTo(sender.ref) val signTxResponse = sender.expectMsgType[SignTransactionResponse] assert(signTxResponse.complete) bitcoinClient.publishTransaction(signTxResponse.tx).pipeTo(sender.ref) @@ -927,7 +1021,7 @@ class BitcoinCoreClientSpec extends TestKitBaseClass with BitcoindService with A test("bump transaction fees with child-pays-for-parent (complex package)") { val sender = TestProbe() - val bitcoinClient = new BitcoinCoreClient(bitcoinrpcclient) + val bitcoinClient = makeBitcoinCoreClient val currentFeerate = FeeratePerKw(500 sat) // We create two separate trees of transactions that will be bumped together: @@ -951,7 +1045,7 @@ class BitcoinCoreClientSpec extends TestKitBaseClass with BitcoindService with A val txNotFunded = Transaction(2, txIn, txOut, 0) bitcoinClient.fundTransaction(txNotFunded, FundTransactionOptions(currentFeerate, changePosition = Some(txOut.length))).pipeTo(sender.ref) val fundTxResponse = sender.expectMsgType[FundTransactionResponse] - bitcoinClient.signTransaction(fundTxResponse.tx, Nil).pipeTo(sender.ref) + signTransaction(bitcoinClient, fundTxResponse.tx, Nil).pipeTo(sender.ref) val signTxResponse = sender.expectMsgType[SignTransactionResponse] assert(signTxResponse.complete) bitcoinClient.publishTransaction(signTxResponse.tx).pipeTo(sender.ref) @@ -1022,7 +1116,7 @@ class BitcoinCoreClientSpec extends TestKitBaseClass with BitcoindService with A test("cannot bump transaction fees (unknown transaction)") { val sender = TestProbe() - val bitcoinClient = new BitcoinCoreClient(bitcoinrpcclient) + val bitcoinClient = makeBitcoinCoreClient bitcoinClient.cpfp(Set(OutPoint(randomBytes32(), 0), OutPoint(randomBytes32(), 3)), FeeratePerKw(1500 sat)).pipeTo(sender.ref) val failure = sender.expectMsgType[Failure] assert(failure.cause.getMessage.contains("some transactions could not be found")) @@ -1030,8 +1124,8 @@ class BitcoinCoreClientSpec extends TestKitBaseClass with BitcoindService with A test("cannot bump transaction fees (invalid outpoint index)") { val sender = TestProbe() - val bitcoinClient = new BitcoinCoreClient(bitcoinrpcclient) - val tx = sendToAddress(getNewAddress(sender), 150_000 sat, sender) + val bitcoinClient = makeBitcoinCoreClient + val tx = sendToAddress(getNewAddress(sender), 150_000 sat) assert(tx.txOut.length == 2) // there must be a change output bitcoinClient.getMempoolTx(tx.txid).pipeTo(sender.ref) val mempoolTx = sender.expectMsgType[MempoolTx] @@ -1045,8 +1139,8 @@ class BitcoinCoreClientSpec extends TestKitBaseClass with BitcoindService with A test("cannot bump transaction fees (transaction already confirmed)") { val sender = TestProbe() - val bitcoinClient = new BitcoinCoreClient(bitcoinrpcclient) - val tx = sendToAddress(getNewAddress(sender), 45_000 sat, sender) + val bitcoinClient = makeBitcoinCoreClient + val tx = sendToAddress(getNewAddress(sender), 45_000 sat) generateBlocks(1) bitcoinClient.cpfp(Set(OutPoint(tx, 0)), FeeratePerKw(2500 sat)).pipeTo(sender.ref) @@ -1056,11 +1150,11 @@ class BitcoinCoreClientSpec extends TestKitBaseClass with BitcoindService with A test("cannot bump transaction fees (non-wallet input)") { val sender = TestProbe() - val bitcoinClient = new BitcoinCoreClient(bitcoinrpcclient) + val bitcoinClient = makeBitcoinCoreClient val txNotFunded = Transaction(2, Nil, TxOut(50_000 sat, Script.pay2wpkh(randomKey().publicKey)) :: Nil, 0) bitcoinClient.fundTransaction(txNotFunded, FundTransactionOptions(FeeratePerKw(1000 sat), changePosition = Some(1))).pipeTo(sender.ref) val fundTxResponse = sender.expectMsgType[FundTransactionResponse] - bitcoinClient.signTransaction(fundTxResponse.tx, Nil).pipeTo(sender.ref) + signTransaction(bitcoinClient, fundTxResponse.tx, Nil).pipeTo(sender.ref) val signTxResponse = sender.expectMsgType[SignTransactionResponse] bitcoinClient.publishTransaction(signTxResponse.tx).pipeTo(sender.ref) sender.expectMsg(signTxResponse.tx.txid) @@ -1072,8 +1166,8 @@ class BitcoinCoreClientSpec extends TestKitBaseClass with BitcoindService with A test("cannot bump transaction fees (amount too low)") { val sender = TestProbe() - val bitcoinClient = new BitcoinCoreClient(bitcoinrpcclient) - val tx = sendToAddress(getNewAddress(sender), 2500 sat, sender) + val bitcoinClient = makeBitcoinCoreClient + val tx = sendToAddress(getNewAddress(sender), 2500 sat) val outputIndex = if (tx.txOut.head.amount == 2500.sat) 0 else 1 bitcoinClient.cpfp(Set(OutPoint(tx, outputIndex)), FeeratePerKw(50_000 sat)).pipeTo(sender.ref) val failure = sender.expectMsgType[Failure] @@ -1082,18 +1176,18 @@ class BitcoinCoreClientSpec extends TestKitBaseClass with BitcoindService with A test("detect if tx has been double-spent") { val sender = TestProbe() - val bitcoinClient = new BitcoinCoreClient(bitcoinrpcclient) + val bitcoinClient = makeBitcoinCoreClient // first let's create a tx val noInputTx1 = Transaction(2, Nil, Seq(TxOut(500_000 sat, Script.pay2wpkh(randomKey().publicKey))), 0) bitcoinClient.fundTransaction(noInputTx1, FundTransactionOptions(FeeratePerKw(2500 sat))).pipeTo(sender.ref) val unsignedTx1 = sender.expectMsgType[FundTransactionResponse].tx - bitcoinClient.signTransaction(unsignedTx1).pipeTo(sender.ref) + signTransaction(bitcoinClient, unsignedTx1).pipeTo(sender.ref) val tx1 = sender.expectMsgType[SignTransactionResponse].tx // let's then generate another tx that double spends the first one val unsignedTx2 = tx1.copy(txOut = Seq(TxOut(tx1.txOut.map(_.amount).sum, Script.pay2wpkh(randomKey().publicKey)))) - bitcoinClient.signTransaction(unsignedTx2).pipeTo(sender.ref) + signTransaction(bitcoinClient, unsignedTx2).pipeTo(sender.ref) val tx2 = sender.expectMsgType[SignTransactionResponse].tx // tx1/tx2 haven't been published, so tx1 isn't double-spent @@ -1120,7 +1214,7 @@ class BitcoinCoreClientSpec extends TestKitBaseClass with BitcoindService with A test("detect if tx has been double-spent (with unconfirmed inputs)") { val sender = TestProbe() - val bitcoinClient = new BitcoinCoreClient(bitcoinrpcclient) + val bitcoinClient = makeBitcoinCoreClient val priv = randomKey() // Let's create one confirmed and one unconfirmed utxo. @@ -1129,7 +1223,7 @@ class BitcoinCoreClientSpec extends TestKitBaseClass with BitcoindService with A val noInputTx = Transaction(2, Nil, Seq(TxOut(amount, Script.pay2wpkh(priv.publicKey))), 0) bitcoinClient.fundTransaction(noInputTx, FundTransactionOptions(FeeratePerKw(2500 sat))).pipeTo(sender.ref) val unsignedTx = sender.expectMsgType[FundTransactionResponse].tx - bitcoinClient.signTransaction(unsignedTx).pipeTo(sender.ref) + signTransaction(bitcoinClient, unsignedTx).pipeTo(sender.ref) sender.expectMsgType[SignTransactionResponse].tx }) bitcoinClient.publishTransaction(txs.head).pipeTo(sender.ref) @@ -1155,7 +1249,7 @@ class BitcoinCoreClientSpec extends TestKitBaseClass with BitcoindService with A { val previousAmountOut = unconfirmedParentTx.txOut.map(_.amount).sum val unsignedTx = unconfirmedParentTx.copy(txOut = Seq(TxOut(previousAmountOut - 50_000.sat, Script.pay2wpkh(randomKey().publicKey)))) - bitcoinClient.signTransaction(unsignedTx).pipeTo(sender.ref) + signTransaction(bitcoinClient, unsignedTx).pipeTo(sender.ref) val signedTx = sender.expectMsgType[SignTransactionResponse].tx bitcoinClient.publishTransaction(signedTx).pipeTo(sender.ref) sender.expectMsg(signedTx.txid) @@ -1182,13 +1276,13 @@ class BitcoinCoreClientSpec extends TestKitBaseClass with BitcoindService with A test("find spending transaction of a given output") { val sender = TestProbe() - val bitcoinClient = new BitcoinCoreClient(bitcoinrpcclient) + val bitcoinClient = makeBitcoinCoreClient bitcoinClient.getBlockHeight().pipeTo(sender.ref) val blockHeight = sender.expectMsgType[BlockHeight] val address = getNewAddress(sender) - val tx1 = sendToAddress(address, 5 btc, sender) + val tx1 = sendToAddress(address, 5 btc) // Transaction is still in the mempool at that point bitcoinClient.getTxConfirmations(tx1.txid).pipeTo(sender.ref) @@ -1230,20 +1324,20 @@ class BitcoinCoreClientSpec extends TestKitBaseClass with BitcoindService with A test("get pubkey for p2wpkh receive address") { val sender = TestProbe() - val bitcoinClient = new BitcoinCoreClient(bitcoinrpcclient) + val bitcoinClient = makeBitcoinCoreClient - // We use Taproot addresses by default (segwit v1). + // eclair onchain key nmanager does not yet support taproot descriptors bitcoinClient.getReceiveAddress().pipeTo(sender.ref) val defaultAddress = sender.expectMsgType[String] val decoded = Bech32.decodeWitnessAddress(defaultAddress) - assert(decoded.getSecond == 1) + assert(decoded.getSecond == 0) // But we can explicitly use segwit v0 addresses. bitcoinClient.getP2wpkhPubkey().pipeTo(sender.ref) val amount = 50 millibtc val receiveKey = sender.expectMsgType[PublicKey] val address = computeP2WpkhAddress(receiveKey, Block.RegtestGenesisBlock.hash) - sendToAddress(address, amount, sender) + sendToAddress(address, amount) generateBlocks(1) bitcoinrpcclient.invoke("getreceivedbyaddress", address).pipeTo(sender.ref) @@ -1253,12 +1347,12 @@ class BitcoinCoreClientSpec extends TestKitBaseClass with BitcoindService with A test("generate segwit change outputs") { val sender = TestProbe() - val bitcoinClient = new BitcoinCoreClient(bitcoinrpcclient) + val bitcoinClient = makeBitcoinCoreClient // Even when we pay a legacy address, our change output must use segwit, otherwise it won't be usable for lightning channels. val pubKey = randomKey().publicKey val legacyAddress = computeP2PkhAddress(pubKey, Block.RegtestGenesisBlock.hash) - bitcoinClient.sendToAddress(legacyAddress, 150_000 sat, 1).pipeTo(sender.ref) + bitcoinClient.sendToPubkeyScript(addressToPublicKeyScript(Block.RegtestGenesisBlock.hash, legacyAddress).toOption.get, 150_000 sat, FeeratePerKw(FeeratePerByte(3.sat))).pipeTo(sender.ref) val txId = sender.expectMsgType[ByteVector32] bitcoinClient.getTransaction(txId).pipeTo(sender.ref) val tx = sender.expectMsgType[Transaction] @@ -1269,12 +1363,14 @@ class BitcoinCoreClientSpec extends TestKitBaseClass with BitcoindService with A } test("does not double-spend inputs of evicted transactions") { + import fr.acinq.bitcoin.scalacompat.KotlinUtils._ + // We fund our wallet with a single confirmed utxo. val sender = TestProbe() val wallet = new BitcoinCoreClient(createWallet("mempool_eviction", sender)) wallet.getP2wpkhPubkey().pipeTo(sender.ref) val walletPubKey = sender.expectMsgType[PublicKey] - val miner = new BitcoinCoreClient(bitcoinrpcclient) + val miner = makeBitcoinCoreClient miner.getP2wpkhPubkey().pipeTo(sender.ref) val nonWalletPubKey = sender.expectMsgType[PublicKey] // We use a large input script to be able to fill the mempool with a few transactions. @@ -1288,7 +1384,7 @@ class BitcoinCoreClientSpec extends TestKitBaseClass with BitcoindService with A val txNotFunded = Transaction(2, Nil, mainOutput +: outputsWithLargeScript, 0) miner.fundTransaction(txNotFunded, FundTransactionOptions(FeeratePerKw(500 sat), changePosition = Some(outputs.length))).pipeTo(sender.ref) val fundedTx = sender.expectMsgType[FundTransactionResponse].tx - miner.signTransaction(fundedTx, allowIncomplete = false).pipeTo(sender.ref) + signTransaction(miner, fundedTx, allowIncomplete = false).pipeTo(sender.ref) val signedTx = sender.expectMsgType[SignTransactionResponse].tx miner.publishTransaction(signedTx).pipeTo(sender.ref) sender.expectMsg(signedTx.txid) @@ -1301,8 +1397,13 @@ class BitcoinCoreClientSpec extends TestKitBaseClass with BitcoindService with A val inputsWithLargeScript = (1 to largeInputsCount).map(i => TxIn(OutPoint(parentTx, i), ByteVector.empty, 0, ScriptWitness(Seq(ByteVector(1), bigInputScript)))) val txIn = mainInput +: inputsWithLargeScript val txOut = Seq(TxOut(amount, Script.pay2wpkh(randomKey().publicKey))) - wallet.signTransaction(Transaction(2, txIn, txOut, 0), allowIncomplete = true).pipeTo(sender.ref) - val signedTx = sender.expectMsgType[SignTransactionResponse].tx + var psbt = new Psbt(Transaction(2, txIn, txOut, 0)) + (1 to largeInputsCount).foreach(i => { + psbt = psbt.updateWitnessInput(OutPoint(parentTx, i), parentTx.txOut(i), null, null, null, psbt.getInput(i).getDerivationPaths).getRight + psbt = psbt.finalizeWitnessInput(i, ScriptWitness(Seq(ByteVector(1), bigInputScript))).getRight + }) + wallet.signPsbt(psbt, Seq(0), Nil).pipeTo(sender.ref) + val signedTx = sender.expectMsgType[ProcessPsbtResponse].finalTx assert(390_000 <= signedTx.weight() && signedTx.weight() <= 400_000) // standard transactions cannot exceed 400 000 WU wallet.publishTransaction(signedTx).pipeTo(sender.ref) sender.expectMsg(signedTx.txid) @@ -1326,4 +1427,82 @@ class BitcoinCoreClientSpec extends TestKitBaseClass with BitcoindService with A assert(sender.expectMsgType[Transaction].txid == tx.txid) } -} \ No newline at end of file +} + +class BitcoinCoreClientWithEclairSignerSpec extends BitcoinCoreClientSpec { + override def useEclairSigner = true + + test("wallets managed by eclair implement BIP84") { + val sender = TestProbe() + val entropy = randomBytes32() + val hex = entropy.toString() + val mnemmonics = MnemonicCode.toMnemonics(entropy) + val seed = MnemonicCode.toSeed(mnemmonics, "") + val master = DeterministicWallet.generate(seed) + + val onchainKeyManager = new LocalOnchainKeyManager(s"eclair_$hex", seed, TimestampSecond.now(), Block.RegtestGenesisBlock.hash) + bitcoinrpcclient.invoke("createwallet", s"eclair_$hex", true, false, "", false, true, true, false).pipeTo(sender.ref) + sender.expectMsgType[JValue] + val jsonRpcClient = new BasicBitcoinJsonRPCClient(rpcAuthMethod = bitcoinrpcauthmethod, host = "localhost", port = bitcoindRpcPort, wallet = Some(s"eclair_$hex")) + importEclairDescriptors(jsonRpcClient, onchainKeyManager) + + // this account xpub can be used to create a watch-only wallet + val accountXPub = DeterministicWallet.encode( + DeterministicWallet.publicKey(DeterministicWallet.derivePrivateKey(master, DeterministicWallet.KeyPath("m/84'/1'/0'"))), + DeterministicWallet.vpub) + assert(onchainKeyManager.getOnchainMasterPubKey(0) == accountXPub) + + val wallet1 = new BitcoinCoreClient(jsonRpcClient, Some(onchainKeyManager)) + + def getBip32Path(address: String): DeterministicWallet.KeyPath = { + wallet1.rpcClient.invoke("getaddressinfo", address).pipeTo(sender.ref) + val JString(bip32path) = (sender.expectMsgType[JValue] \ "hdkeypath") + DeterministicWallet.KeyPath(bip32path) + } + + (0 to 10).foreach { _ => + wallet1.getReceiveAddress().pipeTo(sender.ref) + val address = sender.expectMsgType[String] + val bip32path = getBip32Path(address) + assert(bip32path.path.length == 5 && bip32path.toString().startsWith("m/84'/1'/0'/0")) + assert(computeBIP84Address(DeterministicWallet.derivePrivateKey(master, bip32path).publicKey, Block.RegtestGenesisBlock.hash) == address) + + wallet1.getP2wpkhPubkeyHashForChange().pipeTo(sender.ref) + val Right(changeAddress) = addressFromPublicKeyScript(Block.RegtestGenesisBlock.hash, Script.pay2wpkh(sender.expectMsgType[ByteVector])) + val bip32ChangePath = getBip32Path(changeAddress) + assert(bip32ChangePath.path.length == 5 && bip32ChangePath.toString().startsWith("m/84'/1'/0'/1")) + assert(computeBIP84Address(DeterministicWallet.derivePrivateKey(master, bip32ChangePath).publicKey, Block.RegtestGenesisBlock.hash) == changeAddress) + } + } + + test("use eclair to manage onchain keys") { + val sender = TestProbe() + + (1 to 10).foreach { _ => + val seed = randomBytes32() + val hex = seed.toString() + val onchainKeyManager = new LocalOnchainKeyManager(s"eclair_$hex", seed, TimestampSecond.now(), Block.RegtestGenesisBlock.hash) + bitcoinrpcclient.invoke("createwallet", s"eclair_$hex", true, false, "", false, true, true, false).pipeTo(sender.ref) + sender.expectMsgType[JValue] + + val jsonRpcClient = new BasicBitcoinJsonRPCClient(rpcAuthMethod = bitcoinrpcauthmethod, host = "localhost", port = bitcoindRpcPort, wallet = Some(s"eclair_$hex")) + importEclairDescriptors(jsonRpcClient, onchainKeyManager) + val wallet1 = new BitcoinCoreClient(jsonRpcClient, Some(onchainKeyManager)) + wallet1.getReceiveAddress().pipeTo(sender.ref) + val address = sender.expectMsgType[String] + + // we can send to an onchain address if eclair signs the transactions + sendToAddress(address, 100_0000.sat) + + generateBlocks(1) + + // but bitcoin core's sendtoaddress RPC call will fail because wallets uses an external signer + wallet1.sendToAddress(address, 50_000.sat, 3).pipeTo(sender.ref) + val error = sender.expectMsgType[Failure] + assert(error.cause.getMessage.contains("Private keys are disabled for this wallet")) + + wallet1.sendToPubkeyScript(addressToPublicKeyScript(Block.RegtestGenesisBlock.hash, address).toOption.get, 50_000.sat, FeeratePerKw(FeeratePerByte(5.sat))).pipeTo(sender.ref) + sender.expectMsgType[ByteVector32] + } + } +} diff --git a/eclair-core/src/test/scala/fr/acinq/eclair/blockchain/bitcoind/BitcoindService.scala b/eclair-core/src/test/scala/fr/acinq/eclair/blockchain/bitcoind/BitcoindService.scala index 543500a7f4..3dbcbc01e4 100644 --- a/eclair-core/src/test/scala/fr/acinq/eclair/blockchain/bitcoind/BitcoindService.scala +++ b/eclair-core/src/test/scala/fr/acinq/eclair/blockchain/bitcoind/BitcoindService.scala @@ -19,15 +19,19 @@ package fr.acinq.eclair.blockchain.bitcoind import akka.actor.{Actor, ActorRef, ActorSystem, Props} import akka.pattern.pipe import akka.testkit.{TestKitBase, TestProbe} +import fr.acinq.bitcoin.psbt.Psbt import fr.acinq.bitcoin.scalacompat.Crypto.PrivateKey -import fr.acinq.bitcoin.scalacompat.{Block, Btc, BtcAmount, MilliBtc, Satoshi, Transaction, computeP2WpkhAddress} +import fr.acinq.bitcoin.scalacompat.{Block, Btc, BtcAmount, MilliBtc, Satoshi, Transaction, TxOut, addressToPublicKeyScript, computeP2WpkhAddress} +import fr.acinq.eclair.blockchain.bitcoind.rpc.BitcoinCoreClient.PreviousTx import fr.acinq.eclair.blockchain.bitcoind.rpc.BitcoinJsonRPCAuthMethod.{SafeCookie, UserPassword} -import fr.acinq.eclair.blockchain.bitcoind.rpc.{BasicBitcoinJsonRPCClient, BitcoinJsonRPCAuthMethod, BitcoinJsonRPCClient} -import fr.acinq.eclair.blockchain.fee.{FeeratePerByte, FeeratePerKB} +import fr.acinq.eclair.blockchain.bitcoind.rpc.{BasicBitcoinJsonRPCClient, BitcoinCoreClient, BitcoinJsonRPCAuthMethod, BitcoinJsonRPCClient} +import fr.acinq.eclair.blockchain.fee.{FeeratePerByte, FeeratePerKB, FeeratePerKw} +import fr.acinq.eclair.crypto.keymanager.{LocalOnchainKeyManager, OnchainKeyManager} import fr.acinq.eclair.integration.IntegrationSpec -import fr.acinq.eclair.{BlockHeight, TestUtils, randomKey} +import fr.acinq.eclair.{BlockHeight, TestUtils, TimestampSecond, randomKey} import grizzled.slf4j.Logging import org.json4s.JsonAST._ +import scodec.bits.ByteVector import sttp.client3.okhttp.OkHttpFutureBackend import java.io.File @@ -40,6 +44,8 @@ import scala.io.Source trait BitcoindService extends Logging { self: TestKitBase => + def useEclairSigner: Boolean = false + import BitcoindService._ import scala.sys.process._ @@ -47,7 +53,7 @@ trait BitcoindService extends Logging { implicit val system: ActorSystem implicit val sttpBackend: sttp.client3.SttpBackend[Future, sttp.capabilities.WebSockets] = OkHttpFutureBackend() - val defaultWallet: String = "miner" + val defaultWallet: String = if (useEclairSigner) "eclair" else "miner" val bitcoindPort: Int = TestUtils.availablePort val bitcoindRpcPort: Int = TestUtils.availablePort val bitcoindZmqBlockPort: Int = TestUtils.availablePort @@ -67,7 +73,7 @@ trait BitcoindService extends Logging { var bitcoinrpcclient: BitcoinJsonRPCClient = _ var bitcoinrpcauthmethod: BitcoinJsonRPCAuthMethod = _ var bitcoincli: ActorRef = _ - + val onchainKeyManager = new LocalOnchainKeyManager("eclair", ByteVector.fromValidHex("01" * 32), TimestampSecond.now(), Block.RegtestGenesisBlock.hash) def startBitcoind(useCookie: Boolean = false, defaultAddressType_opt: Option[String] = None, mempoolSize_opt: Option[Int] = None, // mempool size in MB @@ -113,6 +119,8 @@ trait BitcoindService extends Logging { })) } + def makeBitcoinCoreClient: BitcoinCoreClient = new BitcoinCoreClient(bitcoinrpcclient, Some(onchainKeyManager)) + def stopBitcoind(): Unit = { // gracefully stopping bitcoin will make it store its state cleanly to disk, which is good for later debugging val sender = TestProbe() @@ -148,20 +156,35 @@ trait BitcoindService extends Logging { def waitForBitcoindReady(): Unit = { val sender = TestProbe() waitForBitcoindUp(sender) - sender.send(bitcoincli, BitcoinReq("createwallet", defaultWallet)) - sender.expectMsgType[JValue] + if (useEclairSigner) { + // wallet_name, disable_private_keys, blank, passphrase, avoid_reuse, descriptors, load_on_startup, external_signer + bitcoinrpcclient.invoke("createwallet", defaultWallet, true, false, "", false, true, true, false).pipeTo(sender.ref) + sender.expectMsgType[JValue] + + val jsonRpcClient = new BasicBitcoinJsonRPCClient(rpcAuthMethod = bitcoinrpcauthmethod, host = "localhost", port = bitcoindRpcPort, wallet = Some(defaultWallet)) + importEclairDescriptors(jsonRpcClient, onchainKeyManager) + } else { + sender.send(bitcoincli, BitcoinReq("createwallet", defaultWallet)) + sender.expectMsgType[JValue] + } logger.info(s"generating initial blocks to wallet=$defaultWallet...") generateBlocks(150) awaitCond(currentBlockHeight(sender) >= BlockHeight(150), max = 3 minutes, interval = 2 second) } - /** Generate blocks to a given address, or to our wallet if no address is provided. */ + + def importEclairDescriptors(jsonRpcClient: BitcoinJsonRPCClient, keyManager: OnchainKeyManager, probe: TestProbe = TestProbe()): Unit = { + val descriptors = keyManager.getDescriptors(0).descriptors + jsonRpcClient.invoke("importdescriptors", descriptors).pipeTo(probe.ref) + probe.expectMsgType[JValue] + } + def generateBlocks(blockCount: Int, address: Option[String] = None, timeout: FiniteDuration = 10 seconds)(implicit system: ActorSystem): Unit = { val sender = TestProbe() val addressToUse = address match { case Some(addr) => addr case None => - sender.send(bitcoincli, BitcoinReq("getnewaddress")) + sender.send(bitcoincli, BitcoinReq("getnewaddress", "", "bech32")) val JString(address) = sender.expectMsgType[JValue](timeout) address } @@ -198,6 +221,27 @@ trait BitcoindService extends Logging { (priv, address) } + def sendToAddress(address: String, amount: BtcAmount): Transaction = { + import fr.acinq.bitcoin.scalacompat.KotlinUtils._ + + val amountSat = amount match { + case amount: Satoshi => amount + case amount: MilliBtc => amount.toSatoshi + case amount: Btc => amount.toSatoshi + } + val probe = TestProbe() + val tx = Transaction(version = 2, Nil, TxOut(amountSat, addressToPublicKeyScript(Block.RegtestGenesisBlock.hash, address).toOption.get) :: Nil, lockTime = 0) + val client = makeBitcoinCoreClient + val f = for { + funded <- client.fundTransaction(tx, FeeratePerKw(FeeratePerByte(Satoshi(10))), true) + signed <- client.signPsbt(new Psbt(funded.tx), funded.tx.txIn.indices, Nil) + txid <- client.publishTransaction(signed.finalTx) + tx <- client.getTransaction(txid) + } yield tx + f.pipeTo(probe.ref) + probe.expectMsgType[Transaction] + } + /** Send to a given address, without generating blocks to confirm. */ def sendToAddress(address: String, amount: BtcAmount, sender: TestProbe = TestProbe(), rpcClient: BitcoinJsonRPCClient = bitcoinrpcclient): Transaction = { val amountDecimal = amount match { @@ -212,10 +256,19 @@ trait BitcoindService extends Logging { Transaction.read(rawTx) } + def signTransaction(client: BitcoinCoreClient, tx: Transaction): Future[SignTransactionResponse] = signTransaction(client, tx, Nil) + + def signTransaction(client: BitcoinCoreClient, tx: Transaction, allowIncomplete: Boolean): Future[SignTransactionResponse] = signTransaction(client, tx, Nil, allowIncomplete) + + def signTransaction(client: BitcoinCoreClient, tx: Transaction, previousTxs: Seq[PreviousTx], allowIncomplete: Boolean = false): Future[SignTransactionResponse] = { + import fr.acinq.bitcoin.scalacompat.KotlinUtils._ + client.signPsbt(new Psbt(tx), tx.txIn.indices, Nil).map(p => SignTransactionResponse(p.extractFinalTx.getOrElse(p.extractPartiallySignedTx), p.extractFinalTx.isRight)) + } } object BitcoindService { case class BitcoinReq(method: String, params: Any*) + final case class SignTransactionResponse(tx: Transaction, complete: Boolean) } \ No newline at end of file diff --git a/eclair-core/src/test/scala/fr/acinq/eclair/blockchain/bitcoind/ZmqWatcherSpec.scala b/eclair-core/src/test/scala/fr/acinq/eclair/blockchain/bitcoind/ZmqWatcherSpec.scala index 0f3fb250fd..1584316b1f 100644 --- a/eclair-core/src/test/scala/fr/acinq/eclair/blockchain/bitcoind/ZmqWatcherSpec.scala +++ b/eclair-core/src/test/scala/fr/acinq/eclair/blockchain/bitcoind/ZmqWatcherSpec.scala @@ -22,8 +22,9 @@ import akka.actor.{ActorRef, Props, typed} import akka.pattern.pipe import akka.testkit.TestProbe import fr.acinq.bitcoin.scalacompat.{Block, Btc, MilliBtcDouble, OutPoint, SatoshiLong, Script, Transaction, TxOut} -import fr.acinq.eclair.blockchain.OnChainWallet.{FundTransactionResponse, MakeFundingTxResponse, SignTransactionResponse} +import fr.acinq.eclair.blockchain.OnChainWallet.{FundTransactionResponse, MakeFundingTxResponse} import fr.acinq.eclair.blockchain.WatcherSpec._ +import fr.acinq.eclair.blockchain.bitcoind.BitcoindService.SignTransactionResponse import fr.acinq.eclair.blockchain.bitcoind.ZmqWatcher._ import fr.acinq.eclair.blockchain.bitcoind.rpc.BitcoinCoreClient import fr.acinq.eclair.blockchain.bitcoind.rpc.BitcoinCoreClient.FundTransactionOptions @@ -305,7 +306,7 @@ class ZmqWatcherSpec extends TestKitBaseClass with AnyFunSuiteLike with Bitcoind val tx1 = { bitcoinClient.fundTransaction(Transaction(2, Nil, TxOut(150000 sat, Script.pay2wpkh(priv.publicKey)) :: Nil, 0), FundTransactionOptions(FeeratePerKw(250 sat))).pipeTo(probe.ref) val funded = probe.expectMsgType[FundTransactionResponse].tx - bitcoinClient.signTransaction(funded).pipeTo(probe.ref) + signTransaction(bitcoinClient, funded).pipeTo(probe.ref) probe.expectMsgType[SignTransactionResponse].tx } val outputIndex = tx1.txOut.indexWhere(_.publicKeyScript == Script.write(Script.pay2wpkh(priv.publicKey))) diff --git a/eclair-core/src/test/scala/fr/acinq/eclair/blockchain/fee/BitcoinCoreFeeProviderSpec.scala b/eclair-core/src/test/scala/fr/acinq/eclair/blockchain/fee/BitcoinCoreFeeProviderSpec.scala index 1195ca3a55..5987527da7 100644 --- a/eclair-core/src/test/scala/fr/acinq/eclair/blockchain/fee/BitcoinCoreFeeProviderSpec.scala +++ b/eclair-core/src/test/scala/fr/acinq/eclair/blockchain/fee/BitcoinCoreFeeProviderSpec.scala @@ -16,7 +16,6 @@ package fr.acinq.eclair.blockchain.fee -import akka.actor.Status.Failure import akka.pattern.pipe import akka.testkit.TestProbe import fr.acinq.bitcoin.scalacompat._ @@ -35,6 +34,8 @@ import scala.concurrent.{ExecutionContext, Future} class BitcoinCoreFeeProviderSpec extends TestKitBaseClass with BitcoindService with AnyFunSuiteLike with BeforeAndAfterAll with Logging { + override def useEclairSigner = false + override def beforeAll(): Unit = { startBitcoind() waitForBitcoindReady() diff --git a/eclair-core/src/test/scala/fr/acinq/eclair/channel/InteractiveTxBuilderSpec.scala b/eclair-core/src/test/scala/fr/acinq/eclair/channel/InteractiveTxBuilderSpec.scala index ed717f70ba..8641e2bec0 100644 --- a/eclair-core/src/test/scala/fr/acinq/eclair/channel/InteractiveTxBuilderSpec.scala +++ b/eclair-core/src/test/scala/fr/acinq/eclair/channel/InteractiveTxBuilderSpec.scala @@ -21,13 +21,14 @@ import akka.actor.typed.scaladsl.adapter.{ClassicActorSystemOps, actorRefAdapter import akka.pattern.pipe import akka.testkit.TestProbe import com.softwaremill.quicklens.{ModifyPimp, QuicklensAt} +import fr.acinq.bitcoin.psbt.Psbt import fr.acinq.bitcoin.scalacompat.Crypto.PublicKey -import fr.acinq.bitcoin.scalacompat.{ByteVector32, ByteVector64, OP_1, OutPoint, Satoshi, SatoshiLong, Script, ScriptWitness, Transaction, TxOut} -import fr.acinq.eclair.blockchain.OnChainWallet.{FundTransactionResponse, SignTransactionResponse} +import fr.acinq.bitcoin.scalacompat.{Block, ByteVector32, ByteVector64, OP_1, OutPoint, Satoshi, SatoshiLong, Script, ScriptWitness, Transaction, TxOut, addressToPublicKeyScript} +import fr.acinq.eclair.blockchain.OnChainWallet.FundTransactionResponse import fr.acinq.eclair.blockchain.bitcoind.BitcoindService -import fr.acinq.eclair.blockchain.bitcoind.rpc.BitcoinCoreClient.{MempoolTx, Utxo} +import fr.acinq.eclair.blockchain.bitcoind.rpc.BitcoinCoreClient.{MempoolTx, ProcessPsbtResponse, Utxo} import fr.acinq.eclair.blockchain.bitcoind.rpc.{BitcoinCoreClient, BitcoinJsonRPCClient} -import fr.acinq.eclair.blockchain.fee.FeeratePerKw +import fr.acinq.eclair.blockchain.fee.{FeeratePerByte, FeeratePerKw} import fr.acinq.eclair.blockchain.{OnChainWallet, SingleKeyOnChainWallet} import fr.acinq.eclair.channel.fund.InteractiveTxBuilder._ import fr.acinq.eclair.channel.fund.{InteractiveTxBuilder, InteractiveTxSigningSession} @@ -57,9 +58,19 @@ class InteractiveTxBuilderSpec extends TestKitBaseClass with AnyFunSuiteLike wit } private def addUtxo(wallet: BitcoinCoreClient, amount: Satoshi, probe: TestProbe): Unit = { + import fr.acinq.bitcoin.scalacompat.KotlinUtils._ + wallet.getReceiveAddress().pipeTo(probe.ref) val walletAddress = probe.expectMsgType[String] - sendToAddress(walletAddress, amount, probe) + val tx = Transaction(version = 2, Nil, TxOut(amount, addressToPublicKeyScript(Block.RegtestGenesisBlock.hash, walletAddress).toOption.get) :: Nil, lockTime = 0) + val client = makeBitcoinCoreClient + val f = for { + funded <- client.fundTransaction(tx, FeeratePerKw(FeeratePerByte(10.sat)), true) + signed <- client.signPsbt(new Psbt(funded.tx), funded.tx.txIn.indices, Nil) + txid <- client.publishTransaction(signed.finalTx) + } yield txid + f.pipeTo(probe.ref) + probe.expectMsgType[ByteVector32] } private def createInput(channelId: ByteVector32, serialId: UInt64, amount: Satoshi): TxAddInput = { @@ -979,24 +990,25 @@ class InteractiveTxBuilderSpec extends TestKitBaseClass with AnyFunSuiteLike wit val utxosA = Seq(75_000 sat, 60_000 sat) withFixture(fundingA, utxosA, 0 sat, Nil, FeeratePerKw(5000 sat), 660 sat, 0, RequireConfirmedInputs(forLocal = false, forRemote = false)) { f => import f._ + import fr.acinq.bitcoin.scalacompat.KotlinUtils._ // Add some unusable utxos to Alice's wallet. val probe = TestProbe() val legacyTxId = { // Dual funding disallows non-segwit inputs. val legacyAddress = getNewAddress(probe, rpcClientA, Some("legacy")) - sendToAddress(legacyAddress, 100_000 sat, probe).txid + sendToAddress(legacyAddress, 100_000 sat).txid } val bigTxId = { // Dual funding cannot use transactions that exceed 65k bytes. walletA.getP2wpkhPubkey().pipeTo(probe.ref) val publicKey = probe.expectMsgType[PublicKey] val tx = Transaction(2, Nil, TxOut(100_000 sat, Script.pay2wpkh(publicKey)) +: (1 to 2500).map(_ => TxOut(5000 sat, Script.pay2wpkh(randomKey().publicKey))), 0) - val minerWallet = new BitcoinCoreClient(bitcoinrpcclient) + val minerWallet = makeBitcoinCoreClient minerWallet.fundTransaction(tx, FeeratePerKw(500 sat), replaceable = true).pipeTo(probe.ref) val unsignedTx = probe.expectMsgType[FundTransactionResponse].tx - minerWallet.signTransaction(unsignedTx).pipeTo(probe.ref) - val signedTx = probe.expectMsgType[SignTransactionResponse].tx + minerWallet.signPsbt(new Psbt(unsignedTx), unsignedTx.txIn.indices, Nil).pipeTo(probe.ref) + val signedTx = probe.expectMsgType[ProcessPsbtResponse].finalTx assert(Transaction.write(signedTx).length >= 65_000) minerWallet.publishTransaction(signedTx).pipeTo(probe.ref) probe.expectMsgType[ByteVector32] @@ -1032,8 +1044,8 @@ class InteractiveTxBuilderSpec extends TestKitBaseClass with AnyFunSuiteLike wit val legacyTxIds = { // Dual funding disallows non-segwit inputs. val legacyAddress = getNewAddress(probe, rpcClientA, Some("legacy")) - val tx1 = sendToAddress(legacyAddress, 100_000 sat, probe).txid - val tx2 = sendToAddress(legacyAddress, 120_000 sat, probe).txid + val tx1 = sendToAddress(legacyAddress, 100_000 sat).txid + val tx2 = sendToAddress(legacyAddress, 120_000 sat).txid Seq(tx1, tx2) } generateBlocks(1) @@ -1768,8 +1780,8 @@ class InteractiveTxBuilderSpec extends TestKitBaseClass with AnyFunSuiteLike wit // Bob's available utxo is unconfirmed. val probe = TestProbe() - walletB.getReceiveAddress().pipeTo(probe.ref) - walletB.sendToAddress(probe.expectMsgType[String], 75_000 sat, 1).pipeTo(probe.ref) + walletB.getP2wpkhPubkey().pipeTo(probe.ref) + walletB.sendToPubkeyScript(Script.write(Script.pay2wpkh(probe.expectMsgType[PublicKey])), 75_000 sat, FeeratePerKw(FeeratePerByte(1.sat))).pipeTo(probe.ref) probe.expectMsgType[ByteVector32] alice ! Start(alice2bob.ref) @@ -2550,3 +2562,7 @@ class InteractiveTxBuilderSpec extends TestKitBaseClass with AnyFunSuiteLike wit } } + +class InteractiveTxBuilderWithEclairSignerSpec extends InteractiveTxBuilderSpec { + override def useEclairSigner = true +} \ No newline at end of file diff --git a/eclair-core/src/test/scala/fr/acinq/eclair/channel/publish/ReplaceableTxFunderSpec.scala b/eclair-core/src/test/scala/fr/acinq/eclair/channel/publish/ReplaceableTxFunderSpec.scala index e65a69e154..431aa70526 100644 --- a/eclair-core/src/test/scala/fr/acinq/eclair/channel/publish/ReplaceableTxFunderSpec.scala +++ b/eclair-core/src/test/scala/fr/acinq/eclair/channel/publish/ReplaceableTxFunderSpec.scala @@ -70,13 +70,13 @@ class ReplaceableTxFunderSpec extends TestKitBaseClass with AnyFunSuiteLike { paymentHash, 17, ConfirmationTarget.Absolute(BlockHeight(0)) - ), PlaceHolderSig, preimage) + ), PlaceHolderSig, preimage, Nil, Nil) val htlcTimeout = HtlcTimeoutWithWitnessData(HtlcTimeoutTx( InputInfo(OutPoint(commitTx, 1), commitTx.txOut.last, htlcTimeoutScript), Transaction(2, Seq(TxIn(OutPoint(commitTx, 1), ByteVector.empty, 0)), Seq(TxOut(4000 sat, Script.pay2wpkh(PlaceHolderPubKey))), 0), 12, ConfirmationTarget.Absolute(BlockHeight(0)) - ), PlaceHolderSig) + ), PlaceHolderSig, Nil, Nil) (htlcSuccess, htlcTimeout) } @@ -129,7 +129,7 @@ class ReplaceableTxFunderSpec extends TestKitBaseClass with AnyFunSuiteLike { test("adjust previous anchor transaction outputs") { val (commitTx, initialAnchorTx) = createAnchorTx() - val previousAnchorTx = ClaimLocalAnchorWithWitnessData(initialAnchorTx).updateTx(initialAnchorTx.tx.copy( + val previousAnchorTx = ClaimLocalAnchorWithWitnessData(initialAnchorTx, Nil, Nil).updateTx(initialAnchorTx.tx.copy( txIn = Seq( initialAnchorTx.tx.txIn.head, // The previous funding attempt added two wallet inputs: diff --git a/eclair-core/src/test/scala/fr/acinq/eclair/channel/publish/ReplaceableTxPublisherSpec.scala b/eclair-core/src/test/scala/fr/acinq/eclair/channel/publish/ReplaceableTxPublisherSpec.scala index 216e2af3ef..a9fc08575e 100644 --- a/eclair-core/src/test/scala/fr/acinq/eclair/channel/publish/ReplaceableTxPublisherSpec.scala +++ b/eclair-core/src/test/scala/fr/acinq/eclair/channel/publish/ReplaceableTxPublisherSpec.scala @@ -22,12 +22,12 @@ import akka.pattern.pipe import akka.testkit.{TestFSMRef, TestProbe} import com.softwaremill.quicklens.ModifyPimp import fr.acinq.bitcoin.scalacompat.Crypto.PublicKey -import fr.acinq.bitcoin.scalacompat.{BtcAmount, ByteVector32, MilliBtcDouble, OutPoint, SatoshiLong, Transaction} +import fr.acinq.bitcoin.scalacompat.{Block, BtcAmount, ByteVector32, MilliBtcDouble, MnemonicCode, OutPoint, SatoshiLong, Transaction} import fr.acinq.eclair.NotificationsLogger.NotifyNodeOperator import fr.acinq.eclair.blockchain.bitcoind.BitcoindService import fr.acinq.eclair.blockchain.bitcoind.ZmqWatcher._ import fr.acinq.eclair.blockchain.bitcoind.rpc.BitcoinCoreClient.MempoolTx -import fr.acinq.eclair.blockchain.bitcoind.rpc.{BitcoinCoreClient, BitcoinJsonRPCClient} +import fr.acinq.eclair.blockchain.bitcoind.rpc.{BasicBitcoinJsonRPCClient, BitcoinCoreClient, BitcoinJsonRPCClient} import fr.acinq.eclair.blockchain.fee.{ConfirmationTarget, FeeratePerKw, FeeratesPerKw} import fr.acinq.eclair.blockchain.{CurrentBlockHeight, OnchainPubkeyCache} import fr.acinq.eclair.channel._ @@ -36,12 +36,15 @@ import fr.acinq.eclair.channel.publish.ReplaceableTxPublisher.{Publish, Stop, Up import fr.acinq.eclair.channel.publish.TxPublisher.TxRejectedReason._ import fr.acinq.eclair.channel.publish.TxPublisher._ import fr.acinq.eclair.channel.states.{ChannelStateTestsBase, ChannelStateTestsTags} +import fr.acinq.eclair.crypto.keymanager.LocalOnchainKeyManager import fr.acinq.eclair.transactions.Transactions import fr.acinq.eclair.transactions.Transactions._ import fr.acinq.eclair.wire.protocol.{CommitSig, RevokeAndAck} -import fr.acinq.eclair.{BlockHeight, MilliSatoshi, MilliSatoshiLong, NodeParams, NotificationsLogger, TestConstants, TestKitBaseClass, randomKey} +import fr.acinq.eclair.{BlockHeight, MilliSatoshi, MilliSatoshiLong, NodeParams, NotificationsLogger, TestConstants, TestKitBaseClass, TimestampSecond, randomKey} +import org.json4s.JValue import org.scalatest.BeforeAndAfterAll import org.scalatest.funsuite.AnyFunSuiteLike +import scodec.bits.ByteVector import java.util.UUID import java.util.concurrent.atomic.AtomicLong @@ -118,11 +121,8 @@ class ReplaceableTxPublisherSpec extends TestKitBaseClass with AnyFunSuiteLike w } - // NB: we can't use ScalaTest's fixtures, they would see uninitialized bitcoind fields because they sandbox each test. - private def withFixture(utxos: Seq[BtcAmount], channelType: SupportedChannelType)(testFun: Fixture => Any): Unit = { - // Create a unique wallet for this test and ensure it has some btc. - val testId = UUID.randomUUID() - val walletRpcClient = createWallet(s"lightning-$testId") + def createTestWallet(walletName: String) = { + val walletRpcClient = createWallet(walletName) val probe = TestProbe() val walletClient = new BitcoinCoreClient(walletRpcClient) with OnchainPubkeyCache { val pubkey = { @@ -133,11 +133,21 @@ class ReplaceableTxPublisherSpec extends TestKitBaseClass with AnyFunSuiteLike w override def getP2wpkhPubkey(renew: Boolean): PublicKey = pubkey } + (walletRpcClient, walletClient) + } + + // NB: we can't use ScalaTest's fixtures, they would see uninitialized bitcoind fields because they sandbox each test. + private def withFixture(utxos: Seq[BtcAmount], channelType: SupportedChannelType)(testFun: Fixture => Any): Unit = { + // Create a unique wallet for this test and ensure it has some btc. + val testId = UUID.randomUUID() + val (walletRpcClient, walletClient) = createTestWallet(s"lightning-$testId") + val probe = TestProbe() + // Ensure our wallet has some funds. utxos.foreach(amount => { walletClient.getReceiveAddress().pipeTo(probe.ref) val walletAddress = probe.expectMsgType[String] - sendToAddress(walletAddress, amount, probe) + sendToAddress(walletAddress, amount) }) generateBlocks(1) @@ -636,7 +646,7 @@ class ReplaceableTxPublisherSpec extends TestKitBaseClass with AnyFunSuiteLike w // Our wallet receives new unconfirmed utxos: unfortunately, BIP 125 rule #2 doesn't let us use that input... wallet.getReceiveAddress().pipeTo(probe.ref) val walletAddress = probe.expectMsgType[String] - val walletTx = sendToAddress(walletAddress, 5 millibtc, probe) + val walletTx = sendToAddress(walletAddress, 5 millibtc) // A new block is found, and the feerate has increased for our block target, but we can't use our unconfirmed input. system.eventStream.subscribe(probe.ref, classOf[NotifyNodeOperator]) @@ -1639,3 +1649,28 @@ class ReplaceableTxPublisherSpec extends TestKitBaseClass with AnyFunSuiteLike w } } + +class ReplaceableTxPublisherWithEclairSignerSpec extends ReplaceableTxPublisherSpec { + override def createTestWallet(walletName: String) = { + val probe = TestProbe() + // we use the wallet name as a passphrase to make sure we get a new empty wallet + val entropy = ByteVector.fromValidHex("01" * 32) + val seed = MnemonicCode.toSeed(MnemonicCode.toMnemonics(entropy), walletName) + val keyManager = new LocalOnchainKeyManager(walletName, seed, TimestampSecond.now(), Block.RegtestGenesisBlock.hash) + bitcoinrpcclient.invoke("createwallet", walletName, true, false, "", false, true, true, false).pipeTo(probe.ref) + probe.expectMsgType[JValue] + + val walletRpcClient = new BasicBitcoinJsonRPCClient(rpcAuthMethod = bitcoinrpcauthmethod, host = "localhost", port = bitcoindRpcPort, wallet = Some(walletName)) + importEclairDescriptors(walletRpcClient, keyManager) + val walletClient = new BitcoinCoreClient(walletRpcClient, Some(keyManager)) with OnchainPubkeyCache { + val pubkey = { + getP2wpkhPubkey().pipeTo(probe.ref) + probe.expectMsgType[PublicKey] + } + + override def getP2wpkhPubkey(renew: Boolean): PublicKey = pubkey + } + + (walletRpcClient, walletClient) + } +} \ No newline at end of file diff --git a/eclair-core/src/test/scala/fr/acinq/eclair/crypto/keymanager/LocalChannelKeyManagerSpec.scala b/eclair-core/src/test/scala/fr/acinq/eclair/crypto/keymanager/LocalChannelKeyManagerSpec.scala index 936596afe0..3aebdf2783 100644 --- a/eclair-core/src/test/scala/fr/acinq/eclair/crypto/keymanager/LocalChannelKeyManagerSpec.scala +++ b/eclair-core/src/test/scala/fr/acinq/eclair/crypto/keymanager/LocalChannelKeyManagerSpec.scala @@ -16,8 +16,6 @@ package fr.acinq.eclair.crypto.keymanager -import java.io.File -import java.nio.file.Files import fr.acinq.bitcoin.scalacompat.Crypto.{PrivateKey, PublicKey} import fr.acinq.bitcoin.scalacompat.DeterministicWallet.KeyPath import fr.acinq.bitcoin.scalacompat.{Block, ByteVector32, DeterministicWallet} @@ -28,6 +26,9 @@ import fr.acinq.eclair.{NodeParams, TestConstants, TestUtils} import org.scalatest.funsuite.AnyFunSuite import scodec.bits._ +import java.io.File +import java.nio.file.Files + class LocalChannelKeyManagerSpec extends AnyFunSuite { test("generate the same secrets from the same seed") { diff --git a/eclair-core/src/test/scala/fr/acinq/eclair/crypto/keymanager/LocalNodeKeyManagerSpec.scala b/eclair-core/src/test/scala/fr/acinq/eclair/crypto/keymanager/LocalNodeKeyManagerSpec.scala index 89755aa8b7..d6384f4a6e 100644 --- a/eclair-core/src/test/scala/fr/acinq/eclair/crypto/keymanager/LocalNodeKeyManagerSpec.scala +++ b/eclair-core/src/test/scala/fr/acinq/eclair/crypto/keymanager/LocalNodeKeyManagerSpec.scala @@ -16,9 +16,6 @@ package fr.acinq.eclair.crypto.keymanager -import java.io.File -import java.nio.file.Files - import fr.acinq.bitcoin.scalacompat.Crypto.PublicKey import fr.acinq.bitcoin.scalacompat.DeterministicWallet.KeyPath import fr.acinq.bitcoin.scalacompat.{Block, ByteVector32, Crypto} @@ -27,6 +24,9 @@ import fr.acinq.eclair.{NodeParams, TestUtils} import org.scalatest.funsuite.AnyFunSuite import scodec.bits._ +import java.io.File +import java.nio.file.Files + class LocalNodeKeyManagerSpec extends AnyFunSuite { test("generate the same node id from the same seed") { diff --git a/eclair-core/src/test/scala/fr/acinq/eclair/crypto/keymanager/LocalOnchainKeyManagerSpec.scala b/eclair-core/src/test/scala/fr/acinq/eclair/crypto/keymanager/LocalOnchainKeyManagerSpec.scala new file mode 100644 index 0000000000..a59282d7e5 --- /dev/null +++ b/eclair-core/src/test/scala/fr/acinq/eclair/crypto/keymanager/LocalOnchainKeyManagerSpec.scala @@ -0,0 +1,146 @@ +package fr.acinq.eclair.crypto.keymanager + +import fr.acinq.bitcoin.psbt.{KeyPathWithMaster, Psbt} +import fr.acinq.bitcoin.scalacompat.{Block, DeterministicWallet, MnemonicCode, OutPoint, Satoshi, Script, Transaction, TxIn, TxOut} +import fr.acinq.bitcoin.{ScriptFlags, SigHash} +import fr.acinq.eclair.TimestampSecond +import org.scalatest.funsuite.AnyFunSuite +import scodec.bits.ByteVector + +import java.util.Base64 +import scala.jdk.CollectionConverters.SeqHasAsJava + +class LocalOnchainKeyManagerSpec extends AnyFunSuite { + test("sign psbt (non-reg test)") { + val entropy = ByteVector.fromValidHex("01" * 32) + val seed = MnemonicCode.toSeed(MnemonicCode.toMnemonics(entropy), "") + val onchainKeyManager = new LocalOnchainKeyManager("eclair", seed, TimestampSecond.now(), Block.TestnetGenesisBlock.hash) + // data generated by bitcoin core on regtest + val psbt = Psbt.read( + Base64.getDecoder.decode("cHNidP8BAHECAAAAAfZo4nGIyTg77MFmEBkQH1Au3Jl8vzB2WWQGGz/MbyssAAAAAAD9////ArAHPgUAAAAAFgAU6j9yVvLg66Zu3GM/xHbmXT0yvyiAlpgAAAAAABYAFODscQh3N7lmDYyV5yrHpGL2Zd4JAAAAAAABAH0CAAAAAaNdmqUNlziIjSaif3JUcvJWdyF0U5bYq13NMe+LbaBZAAAAAAD9////AjSp1gUAAAAAFgAUjfFMfBg8ulo/874n3+0ode7ka0BAQg8AAAAAACIAIPUn/XU17DfnvDkj8gn2twG3jtr2Z7sthy9K2MPTdYkaAAAAAAEBHzSp1gUAAAAAFgAUjfFMfBg8ulo/874n3+0ode7ka0AiBgM+PDdyxsVisa66SyBxiUvhEam8lEP64yujvVsEcGaqIxgPCfOBVAAAgAEAAIAAAACAAQAAAAMAAAAAIgIDWmAhb/sCV9+HjwFpPuy2TyEBi/Y11wrEHZUihe3N80EYDwnzgVQAAIABAACAAAAAgAEAAAAFAAAAAAA=") + ).getRight + + val psbt1 = onchainKeyManager.signPsbt(psbt, psbt.getInputs.toArray().indices, Seq(0)) + val tx = psbt1.extract() + assert(tx.isRight) + } + + test("sign psbt") { + import fr.acinq.bitcoin.scalacompat.KotlinUtils._ + import fr.acinq.bitcoin.utils.EitherKt + + val seed = ByteVector.fromValidHex("01" * 32) + val onchainKeyManager = new LocalOnchainKeyManager("eclair", seed, TimestampSecond.now(), Block.TestnetGenesisBlock.hash) + + // create a watch-only BIP84 wallet from our key manager xpub + val (_, accountPub) = DeterministicWallet.ExtendedPublicKey.decode(onchainKeyManager.getOnchainMasterPubKey(0)) + val mainPub = DeterministicWallet.derivePublicKey(accountPub, 0) + val changePub = DeterministicWallet.derivePublicKey(accountPub, 1) + + def getPublicKey(index: Long) = DeterministicWallet.derivePublicKey(mainPub, index).publicKey + + val utxos = Seq( + Transaction(version = 2, txIn = Nil, txOut = TxOut(Satoshi(1000_000), Script.pay2wpkh(getPublicKey(0))) :: Nil, lockTime = 0), + Transaction(version = 2, txIn = Nil, txOut = TxOut(Satoshi(1100_000), Script.pay2wpkh(getPublicKey(1))) :: Nil, lockTime = 0), + Transaction(version = 2, txIn = Nil, txOut = TxOut(Satoshi(1200_000), Script.pay2wpkh(getPublicKey(2))) :: Nil, lockTime = 0), + ) + val bip32paths = Seq( + new KeyPathWithMaster(0, new fr.acinq.bitcoin.KeyPath("m/84'/1'/0'/0/0")), + new KeyPathWithMaster(0, new fr.acinq.bitcoin.KeyPath("m/84'/1'/0'/0/1")), + new KeyPathWithMaster(0, new fr.acinq.bitcoin.KeyPath("m/84'/1'/0'/0/2")), + ) + + val tx = Transaction(version = 2, + txIn = utxos.map(tx => TxIn(OutPoint(tx, 0), Nil, fr.acinq.bitcoin.TxIn.SEQUENCE_FINAL)), + txOut = TxOut(Satoshi(1000_000), Script.pay2wpkh(getPublicKey(0))) :: Nil, lockTime = 0) + val psbt: Psbt = { + val p0 = new Psbt(tx).updateWitnessInput(OutPoint(utxos(0), 0), utxos(0).txOut(0), null, Script.pay2pkh(getPublicKey(0)).map(scala2kmp).asJava, null, java.util.Map.of(getPublicKey(0), bip32paths(0))) + val p1 = EitherKt.flatMap(p0, (psbt: Psbt) => psbt.updateWitnessInput(OutPoint(utxos(1), 0), utxos(1).txOut(0), null, Script.pay2pkh(getPublicKey(1)).map(scala2kmp).asJava, null, java.util.Map.of(getPublicKey(1), bip32paths(1)))) + val p2 = EitherKt.flatMap(p1, (psbt: Psbt) => psbt.updateWitnessInput(OutPoint(utxos(2), 0), utxos(2).txOut(0), null, Script.pay2pkh(getPublicKey(2)).map(scala2kmp).asJava, null, java.util.Map.of(getPublicKey(2), bip32paths(2)))) + val p3 = EitherKt.flatMap(p2, (psbt: Psbt) => psbt.updateNonWitnessInput(utxos(0), 0, null, null, java.util.Map.of())) + val p4 = EitherKt.flatMap(p3, (psbt: Psbt) => psbt.updateNonWitnessInput(utxos(1), 0, null, null, java.util.Map.of())) + val p5 = EitherKt.flatMap(p4, (psbt: Psbt) => psbt.updateNonWitnessInput(utxos(2), 0, null, null, java.util.Map.of())) + val p6 = EitherKt.flatMap(p5, (psbt: Psbt) => psbt.updateWitnessOutput(0, null, null, java.util.Map.of(getPublicKey(0), bip32paths(0)))) + p6.getRight + } + + { + // sign all inputs and outputs + val psbt1 = onchainKeyManager.signPsbt(psbt, Seq(0, 1, 2), Seq(0)) + val signedTx = psbt1.extract().getRight + Transaction.correctlySpends(signedTx, utxos, ScriptFlags.STANDARD_SCRIPT_VERIFY_FLAGS) + } + { + // sign the first 2 inputs only + val psbt1 = onchainKeyManager.signPsbt(psbt, Seq(0, 1), Seq(0)) + // extracting the final tx fails because no all inputs as signed + assert(psbt1.extract().isLeft) + assert(psbt1.getInput(2).getScriptWitness == null) + } + { + // provide a wrong derivation path for the first input + val updated = psbt.updateWitnessInput(OutPoint(utxos(0), 0), utxos(0).txOut(0), null, Script.pay2pkh(getPublicKey(0)).map(scala2kmp).asJava, null, java.util.Map.of(getPublicKey(0), bip32paths(2))).getRight // wrong bip32 path + val error = intercept[IllegalArgumentException] { + onchainKeyManager.signPsbt(updated, Seq(0, 1, 2), Seq(0)) + } + assert(error.getMessage.contains("cannot compute private key")) + } + { + // provide a wrong derivation path for the first output + val updated = psbt.updateWitnessOutput(0, null, null, java.util.Map.of(getPublicKey(0), bip32paths(1))).getRight // wrong path + val error = intercept[IllegalArgumentException] { + onchainKeyManager.signPsbt(updated, Seq(0, 1, 2), Seq(0)) + } + assert(error.getMessage.contains("cannot compute public key")) + } + { + // lie about the amount being spent + val updated = psbt.updateWitnessInput(OutPoint(utxos(0), 0), utxos(0).txOut(0).copy(amount = Satoshi(10)), null, Script.pay2pkh(getPublicKey(0)).map(scala2kmp).asJava, null, java.util.Map.of(getPublicKey(0), bip32paths(0))).getRight + val error = intercept[IllegalArgumentException] { + onchainKeyManager.signPsbt(updated, Seq(0, 1, 2), Seq(0)) + } + assert(error.getMessage.contains("utxo mismatch")) + } + { + // do not provide non-witness utxo for utxo #2 + val psbt: Psbt = { + val p0 = new Psbt(tx).updateWitnessInput(OutPoint(utxos(0), 0), utxos(0).txOut(0), null, Script.pay2pkh(getPublicKey(0)).map(scala2kmp).asJava, null, java.util.Map.of(getPublicKey(0), bip32paths(0))) + val p1 = EitherKt.flatMap(p0, (psbt: Psbt) => psbt.updateWitnessInput(OutPoint(utxos(1), 0), utxos(1).txOut(0), null, Script.pay2pkh(getPublicKey(1)).map(scala2kmp).asJava, null, java.util.Map.of(getPublicKey(1), bip32paths(1)))) + val p2 = EitherKt.flatMap(p1, (psbt: Psbt) => psbt.updateWitnessInput(OutPoint(utxos(2), 0), utxos(2).txOut(0), null, Script.pay2pkh(getPublicKey(2)).map(scala2kmp).asJava, null, java.util.Map.of(getPublicKey(2), bip32paths(2)))) + val p3 = EitherKt.flatMap(p2, (psbt: Psbt) => psbt.updateNonWitnessInput(utxos(0), 0, null, null, java.util.Map.of())) + val p4 = EitherKt.flatMap(p3, (psbt: Psbt) => psbt.updateNonWitnessInput(utxos(1), 0, null, null, java.util.Map.of())) + val p5 = EitherKt.flatMap(p4, (psbt: Psbt) => psbt.updateWitnessOutput(0, null, null, java.util.Map.of(getPublicKey(0), bip32paths(0)))) + p5.getRight + } + val error = intercept[IllegalArgumentException] { + onchainKeyManager.signPsbt(psbt, Seq(0, 1, 2), Seq(0)) + } + assert(error.getMessage.contains("non-witness utxo is missing")) + } + { + // use sighash type != SIGHASH_ALL + val updated = psbt.updateWitnessInput(OutPoint(utxos(0), 0), utxos(0).txOut(0), null, Script.pay2pkh(getPublicKey(0)).map(scala2kmp).asJava, SigHash.SIGHASH_SINGLE, java.util.Map.of(getPublicKey(0), bip32paths(0))).getRight + val error = intercept[IllegalArgumentException] { + onchainKeyManager.signPsbt(updated, Seq(0, 1, 2), Seq(0)) + } + assert(error.getMessage.contains("input sighashtype must be SIGHASH_ALL")) + } + } + + test("compute descriptor checksums") { + val data = Seq( + "pkh([6ded4eb8/44h/0h/0h]xpub6C6N5WVF5zmurBR52MZZj8Jxm6eDiKyM4wFCm7xTYBEsAvJPqBKp2u2K7RTsZaYDN8duBWq4acrD4vrwjaKHTYuntGjL334nVHtLNuaj5Mu/0/*)#5mzpq0w6", + "wpkh([6ded4eb8/84h/0h/0h]xpub6CDeom4xT3Wg7BuyXU2Sd9XerTKttyfxRwJE36mi5HxFYpYdtdwM76Zx8swPnc6zxuArMYJgjNy91fJ13YtGPHgf49YqA8KdXg6D69tzNFh/0/*)#refya6f0", + "sh(wpkh([6ded4eb8/49h/0h/0h]xpub6Cb8jR9kYsfC6kj9CsE18SyudWjW2V3FnBFkT2oqq6n7NWWvJrjhFin3sAYg8X7ApX8iPophBa98mo4nMvSxnqrXvpnwaRopecQz859Ai1s/0/*))#xrhyhtvl", + "tr([6ded4eb8/86h/0h/0h]xpub6CDp1iw76taes3pkqfiJ6PYhwURkaYksJ62CrrdTVr6ow9wR9mKAtUGoZQqb8pRDiq2F8k31tYrrJjVGTRSLYGQ7nYpmewH94ThsAgDxJ4h/0/*)#2nm7drky", + "pkh([6ded4eb8/44h/0h/0h]xpub6C6N5WVF5zmurBR52MZZj8Jxm6eDiKyM4wFCm7xTYBEsAvJPqBKp2u2K7RTsZaYDN8duBWq4acrD4vrwjaKHTYuntGjL334nVHtLNuaj5Mu/1/*)#908qa67z", + "wpkh([6ded4eb8/84h/0h/0h]xpub6CDeom4xT3Wg7BuyXU2Sd9XerTKttyfxRwJE36mi5HxFYpYdtdwM76Zx8swPnc6zxuArMYJgjNy91fJ13YtGPHgf49YqA8KdXg6D69tzNFh/1/*)#jdv9q0eh", + "sh(wpkh([6ded4eb8/49h/0h/0h]xpub6Cb8jR9kYsfC6kj9CsE18SyudWjW2V3FnBFkT2oqq6n7NWWvJrjhFin3sAYg8X7ApX8iPophBa98mo4nMvSxnqrXvpnwaRopecQz859Ai1s/1/*))#nzej05eq", + "tr([6ded4eb8/86h/0h/0h]xpub6CDp1iw76taes3pkqfiJ6PYhwURkaYksJ62CrrdTVr6ow9wR9mKAtUGoZQqb8pRDiq2F8k31tYrrJjVGTRSLYGQ7nYpmewH94ThsAgDxJ4h/1/*)#m87lskxu" + ) + data.foreach(dnc => { + val Array(desc, checksum) = dnc.split('#') + assert(checksum == LocalOnchainKeyManager.descriptorChecksum(desc)) + }) + } +} diff --git a/eclair-core/src/test/scala/fr/acinq/eclair/integration/ChannelIntegrationSpec.scala b/eclair-core/src/test/scala/fr/acinq/eclair/integration/ChannelIntegrationSpec.scala index 95261618a7..29adfe7718 100644 --- a/eclair-core/src/test/scala/fr/acinq/eclair/integration/ChannelIntegrationSpec.scala +++ b/eclair-core/src/test/scala/fr/acinq/eclair/integration/ChannelIntegrationSpec.scala @@ -37,7 +37,7 @@ import fr.acinq.eclair.router.Router import fr.acinq.eclair.transactions.Transactions.{AnchorOutputsCommitmentFormat, TxOwner} import fr.acinq.eclair.transactions.{OutgoingHtlc, Scripts, Transactions} import fr.acinq.eclair.wire.protocol._ -import fr.acinq.eclair.{MilliSatoshi, MilliSatoshiLong, randomBytes32} +import fr.acinq.eclair.{MilliSatoshi, MilliSatoshiLong, TestUtils, randomBytes32} import org.json4s.JsonAST.{JString, JValue} import java.util.UUID @@ -455,9 +455,12 @@ abstract class ChannelIntegrationSpec extends IntegrationSpec { class StandardChannelIntegrationSpec extends ChannelIntegrationSpec { test("start eclair nodes") { - instantiateEclairNode("A", ConfigFactory.parseMap(Map("eclair.node-alias" -> "A", "eclair.channel.expiry-delta-blocks" -> 40, "eclair.channel.fulfill-safety-before-timeout-blocks" -> 12, "eclair.server.port" -> 29740, "eclair.api.port" -> 28090).asJava).withFallback(withDefaultCommitment).withFallback(commonConfig)) - instantiateEclairNode("C", ConfigFactory.parseMap(Map("eclair.node-alias" -> "C", "eclair.channel.expiry-delta-blocks" -> 40, "eclair.channel.fulfill-safety-before-timeout-blocks" -> 12, "eclair.server.port" -> 29741, "eclair.api.port" -> 28091).asJava).withFallback(withAnchorOutputs).withFallback(commonConfig)) - instantiateEclairNode("F", ConfigFactory.parseMap(Map("eclair.node-alias" -> "F", "eclair.channel.expiry-delta-blocks" -> 40, "eclair.channel.fulfill-safety-before-timeout-blocks" -> 12, "eclair.server.port" -> 29742, "eclair.api.port" -> 28092).asJava).withFallback(withDefaultCommitment).withFallback(commonConfig)) + val mapA = Map("eclair.node-alias" -> "A", "eclair.channel.expiry-delta-blocks" -> 40, "eclair.channel.fulfill-safety-before-timeout-blocks" -> 12, "eclair.server.port" -> TestUtils.availablePort, "eclair.api.port" -> TestUtils.availablePort) + val mapB = Map("eclair.node-alias" -> "C", "eclair.channel.expiry-delta-blocks" -> 40, "eclair.channel.fulfill-safety-before-timeout-blocks" -> 12, "eclair.server.port" -> TestUtils.availablePort, "eclair.api.port" -> TestUtils.availablePort) + val mapC = Map("eclair.node-alias" -> "F", "eclair.channel.expiry-delta-blocks" -> 40, "eclair.channel.fulfill-safety-before-timeout-blocks" -> 12, "eclair.server.port" -> TestUtils.availablePort, "eclair.api.port" -> TestUtils.availablePort) + instantiateEclairNode("A", ConfigFactory.parseMap(mapA.asJava).withFallback(withDefaultCommitment).withFallback(commonConfig)) + instantiateEclairNode("C", ConfigFactory.parseMap(mapB.asJava).withFallback(withAnchorOutputs).withFallback(commonConfig)) + instantiateEclairNode("F", ConfigFactory.parseMap(mapC.asJava).withFallback(withDefaultCommitment).withFallback(commonConfig)) } test("connect nodes") { @@ -628,6 +631,10 @@ class StandardChannelIntegrationSpec extends ChannelIntegrationSpec { } +class StandardChannelIntegrationWithEclairSignerSpec extends StandardChannelIntegrationSpec { + override def useEclairSigner: Boolean = true +} + abstract class AnchorChannelIntegrationSpec extends ChannelIntegrationSpec { val commitmentFormat: AnchorOutputsCommitmentFormat diff --git a/eclair-core/src/test/scala/fr/acinq/eclair/integration/IntegrationSpec.scala b/eclair-core/src/test/scala/fr/acinq/eclair/integration/IntegrationSpec.scala index cd1e533041..dd6baf7413 100644 --- a/eclair-core/src/test/scala/fr/acinq/eclair/integration/IntegrationSpec.scala +++ b/eclair-core/src/test/scala/fr/acinq/eclair/integration/IntegrationSpec.scala @@ -28,7 +28,7 @@ import fr.acinq.eclair.payment.relay.Relayer.RelayFees import fr.acinq.eclair.router.Graph.WeightRatios import fr.acinq.eclair.router.RouteCalculation.ROUTE_MAX_LENGTH import fr.acinq.eclair.router.Router.{MultiPartParams, PathFindingConf, SearchBoundaries, NORMAL => _, State => _} -import fr.acinq.eclair.{BlockHeight, CltvExpiryDelta, Kit, MilliSatoshi, MilliSatoshiLong, Setup, TestKitBaseClass} +import fr.acinq.eclair.{BlockHeight, CltvExpiryDelta, Kit, MilliSatoshi, MilliSatoshiLong, Setup, TestKitBaseClass, randomBytes32} import grizzled.slf4j.Logging import org.json4s.{DefaultFormats, Formats} import org.scalatest.BeforeAndAfterAll @@ -147,7 +147,7 @@ abstract class IntegrationSpec extends TestKitBaseClass with BitcoindService wit val datadir = new File(INTEGRATION_TMP_DIR, s"datadir-eclair-$name") datadir.mkdirs() implicit val system: ActorSystem = ActorSystem(s"system-$name", config) - val setup = new Setup(datadir, pluginParams = Seq.empty) + val setup = new Setup(datadir, pluginParams = Seq.empty, seeds_opt = Some(Setup.Seeds(randomBytes32(), randomBytes32())), onchainKeyManager_opt = Some(onchainKeyManager)) val kit = Await.result(setup.bootstrap, 10 seconds) nodes = nodes + (name -> kit) } diff --git a/eclair-core/src/test/scala/fr/acinq/eclair/integration/PaymentIntegrationSpec.scala b/eclair-core/src/test/scala/fr/acinq/eclair/integration/PaymentIntegrationSpec.scala index ea0604ab72..cab4e4735e 100644 --- a/eclair-core/src/test/scala/fr/acinq/eclair/integration/PaymentIntegrationSpec.scala +++ b/eclair-core/src/test/scala/fr/acinq/eclair/integration/PaymentIntegrationSpec.scala @@ -840,7 +840,7 @@ class PaymentIntegrationSpec extends IntegrationSpec { if (i % 10 == 0) { generateBlocks(1, Some(address)) } - AnnouncementsBatchValidationSpec.simulateChannel(bitcoinClient) + AnnouncementsBatchValidationSpec.simulateChannel(bitcoinClient, onchainKeyManager) } generateBlocks(1, Some(address)) logger.info(s"simulated ${channels.size} channels") diff --git a/eclair-core/src/test/scala/fr/acinq/eclair/integration/basic/fixtures/MinimalNodeFixture.scala b/eclair-core/src/test/scala/fr/acinq/eclair/integration/basic/fixtures/MinimalNodeFixture.scala index b82ecc8dba..6fb7853f27 100644 --- a/eclair-core/src/test/scala/fr/acinq/eclair/integration/basic/fixtures/MinimalNodeFixture.scala +++ b/eclair-core/src/test/scala/fr/acinq/eclair/integration/basic/fixtures/MinimalNodeFixture.scala @@ -17,7 +17,7 @@ import fr.acinq.eclair.blockchain.fee.{FeeratePerKw, FeeratesPerKw} import fr.acinq.eclair.channel._ import fr.acinq.eclair.channel.fsm.Channel import fr.acinq.eclair.crypto.TransportHandler -import fr.acinq.eclair.crypto.keymanager.{LocalChannelKeyManager, LocalNodeKeyManager} +import fr.acinq.eclair.crypto.keymanager.{LocalChannelKeyManager, LocalNodeKeyManager, LocalOnchainKeyManager} import fr.acinq.eclair.io.Peer.OpenChannelResponse import fr.acinq.eclair.io.PeerConnection.ConnectionResult import fr.acinq.eclair.io.{Peer, PeerConnection, PendingChannelsRateLimiter, Switchboard} diff --git a/eclair-core/src/test/scala/fr/acinq/eclair/router/AnnouncementsBatchValidationSpec.scala b/eclair-core/src/test/scala/fr/acinq/eclair/router/AnnouncementsBatchValidationSpec.scala index b5d7775a4f..093a8397ba 100644 --- a/eclair-core/src/test/scala/fr/acinq/eclair/router/AnnouncementsBatchValidationSpec.scala +++ b/eclair-core/src/test/scala/fr/acinq/eclair/router/AnnouncementsBatchValidationSpec.scala @@ -19,18 +19,19 @@ package fr.acinq.eclair.router import akka.actor.ActorSystem import akka.pattern.pipe import akka.testkit.TestProbe -import sttp.client3.okhttp.OkHttpFutureBackend import fr.acinq.bitcoin.scalacompat.Crypto.PrivateKey import fr.acinq.bitcoin.scalacompat.{Block, Satoshi, SatoshiLong, Script, Transaction} import fr.acinq.eclair.blockchain.bitcoind.ZmqWatcher.ValidateResult import fr.acinq.eclair.blockchain.bitcoind.rpc.BitcoinJsonRPCAuthMethod.UserPassword import fr.acinq.eclair.blockchain.bitcoind.rpc.{BasicBitcoinJsonRPCClient, BitcoinCoreClient} import fr.acinq.eclair.blockchain.fee.FeeratePerKw +import fr.acinq.eclair.crypto.keymanager.OnchainKeyManager import fr.acinq.eclair.transactions.Scripts import fr.acinq.eclair.wire.protocol.{ChannelAnnouncement, ChannelUpdate} import fr.acinq.eclair.{BlockHeight, CltvExpiryDelta, Features, MilliSatoshiLong, RealShortChannelId, ShortChannelId, randomKey} import org.json4s.JsonAST.JString import org.scalatest.funsuite.AnyFunSuite +import sttp.client3.okhttp.OkHttpFutureBackend import scala.concurrent.duration._ import scala.concurrent.{Await, ExecutionContext} @@ -54,7 +55,7 @@ class AnnouncementsBatchValidationSpec extends AnyFunSuite { val channels = for (i <- 0 until 50) yield { // let's generate a block every 10 txs so that we can compute short ids if (i % 10 == 0) generateBlocks(1, bitcoinClient) - simulateChannel(bitcoinClient) + simulateChannel(bitcoinClient, null) } generateBlocks(6, bitcoinClient) val announcements = channels.map(c => makeChannelAnnouncement(c, bitcoinClient)) @@ -85,7 +86,7 @@ object AnnouncementsBatchValidationSpec { Await.result(generatedF, 10 seconds) } - def simulateChannel(bitcoinClient: BitcoinCoreClient)(implicit ec: ExecutionContext): SimulatedChannel = { + def simulateChannel(bitcoinClient: BitcoinCoreClient, onchainKeyManager: OnchainKeyManager)(implicit ec: ExecutionContext): SimulatedChannel = { val node1Key = randomKey() val node2Key = randomKey() val node1BitcoinKey = randomKey() diff --git a/eclair-node/src/main/scala/fr/acinq/eclair/api/handlers/OnChain.scala b/eclair-node/src/main/scala/fr/acinq/eclair/api/handlers/OnChain.scala index 876b2305f7..9ed4366d05 100644 --- a/eclair-node/src/main/scala/fr/acinq/eclair/api/handlers/OnChain.scala +++ b/eclair-node/src/main/scala/fr/acinq/eclair/api/handlers/OnChain.scala @@ -22,6 +22,7 @@ import fr.acinq.eclair.api.Service import fr.acinq.eclair.api.directives.EclairDirectives import fr.acinq.eclair.api.serde.FormParamExtractors._ import fr.acinq.eclair.blockchain.fee.FeeratePerByte +import org.json4s.{JObject, JString} trait OnChain { this: Service with EclairDirectives => @@ -64,6 +65,21 @@ trait OnChain { complete(eclairApi.globalBalance()) } - val onChainRoutes: Route = getNewAddress ~ sendOnChain ~ cpfpBumpFees ~ onChainBalance ~ onChainTransactions ~ globalBalance + val getmasterxpub: Route = postRequest("getmasterxpub") { implicit t => + formFields("account".as[Long]) { account => + val xpub = this.eclairApi.getOnchainMasterPubKey(account) + complete(new JObject(List("xpub" -> JString(xpub)))) + } + } + + val getdescriptors: Route = postRequest("getdescriptors") { implicit t => + formFields("account".as[Long].?) { + (account_opt) => + val descriptors = this.eclairApi.getDescriptors(account_opt.getOrElse(0L)) + complete(descriptors.descriptors) + } + } + + val onChainRoutes: Route = getNewAddress ~ sendOnChain ~ cpfpBumpFees ~ onChainBalance ~ onChainTransactions ~ globalBalance ~ getmasterxpub ~ getdescriptors } From 9f646419c3b6de5ff27288240d18b13bff535cbe Mon Sep 17 00:00:00 2001 From: sstone Date: Wed, 21 Jun 2023 16:32:11 +0200 Subject: [PATCH 2/8] Create eclair-backed wallet automatically on startup When eclair starts, if it is configured to manage bitcoin core's onchain key, and the configured wallet does not exist yet, and eclair descriptor's timestamps are less then 2 hours old, eclair will automatically create the configured wallet with the appropriate options and import its descriptors. --- docs/BitcoinCoreKeys.md | 120 +++++++++++------- docs/Configure.md | 2 +- docs/release-notes/eclair-vnext.md | 6 +- eclair-core/src/main/resources/reference.conf | 1 - .../main/scala/fr/acinq/eclair/Eclair.scala | 15 ++- .../main/scala/fr/acinq/eclair/Setup.scala | 23 +++- .../bitcoind/rpc/BitcoinCoreClient.scala | 31 ++++- .../keymanager/LocalOnchainKeyManager.scala | 4 +- .../bitcoind/BitcoinCoreClientSpec.scala | 15 +-- .../blockchain/bitcoind/BitcoindService.scala | 40 +++--- .../publish/ReplaceableTxPublisherSpec.scala | 11 +- .../eclair/integration/IntegrationSpec.scala | 7 +- .../acinq/eclair/api/handlers/OnChain.scala | 16 ++- 13 files changed, 185 insertions(+), 106 deletions(-) diff --git a/docs/BitcoinCoreKeys.md b/docs/BitcoinCoreKeys.md index 9040f45c71..4f9eeb2617 100644 --- a/docs/BitcoinCoreKeys.md +++ b/docs/BitcoinCoreKeys.md @@ -1,11 +1,14 @@ # Using Eclair to manage your Bitcoin Core wallet's private keys -You can configure Eclair to control (and never expose) the private keys of your Bitcoin Core wallet. This is very useful if your Bitcoin and Eclair nodes run on different machines for example, with a setup for the Bitcoin host that +You can configure Eclair to control (and never expose) the private keys of your Bitcoin Core wallet. This feature was designed to take advantage of deployements where your Eclair node runs in a +"trusted" runtime environment, but is also very useful if your Bitcoin and Eclair nodes run on different machines for example, with a setup for the Bitcoin host that is less secure than for Eclair (because it is shared among several services for example). +## Configuring Eclair and Bitcoin Core to use a new Eclair-backed bitcoin wallet + Follow these steps to delegate onchain key management to eclair: -1) Generate or import a BIP39 mnemonic code and passphrase +1) Generate a BIP39 mnemonic code and passphrase You can use any BIP39-compatible tool, including most hardware wallets. @@ -24,73 +27,94 @@ This is an example of `eclair-signer.conf` configuration file: ```hocon { - eclair { - signer { - wallet = "eclair" - mnemonics = "legal winner thank year wave sausage worth useful legal winner thank year wave sausage worth useful legal winner thank year wave sausage worth title" - passphrase = "" - timestamp = 1686055705 - } - } + eclair { + signer { + wallet = "eclair" + mnemonics = "legal winner thank year wave sausage worth useful legal winner thank year wave sausage worth useful legal winner thank year wave sausage worth title" + passphrase = "" + timestamp = 1686055705 + } + } } ``` -You must set `eclair.signer.wallet` to a name that is different from your current Bitcoin Core wallet. +3) Configure Eclair to handle private keys for this wallet + +Set `eclair.bitcoind.wallet` to the name of the wallet in your `eclair-signer.conf` file (`eclair` in the example above) and restart Eclair. +Eclair will automatically create a new, empty, descriptor-enabled, watch-only wallet in Bitcoin Core and import its descriptors. + +:warning: Eclair will not import descriptors if the timestamp set in your `eclair-signer.conf` is more than 2 hours old. If the mnmeonics and +passphrase that your are using are new, you can safely update this timestamp, but if they have been used before you will need to follow +the steps described in the next section. + +You now have a Bitcoin Core watch-only wallet for which only your Eclair node can sign transactions. This Bitcoin Core wallet can +safely be copied to another Bitcoin Core node to monitor your onchain funds. + +You can also use `eclair-cli getmasterxpub` to get a BIP32 extended public key that you can import into any compatible Bitcoin wallet +to create a watch-only wallet (Electrum for example) that you can use to monitor your Bitcoin Core balance. + +:warning: this means that your Bitcoin Core wallet cannot send funds on its own (since it cannot access private keys to sign transactions). +To send funds onchain you must use `eclair-cli sendonchain`. + +:warning: to backup the private keys of this wallet you must either backup your mnemonic code and passphrase, or backup the `eclair-signer.conf` file in your eclair +directory (default is `~/.eclair`) along with your channels and node seed files. + +:warning: You can also initialise a backup onchain wallet with the same mnemonic code and passphrase (on a hardware wallet for example), but be warned that using them may interfere with your node's operations (for example you may end up +double-spending funding transactions generated by your node). + +## Importing an existing Eclair-backed bitcoin core wallet + +The steps above describe how you can simply switch to a new Eclair-backed bitcoin wallet. +If you are already using an Eclair-backed bitcoin wallet that you want to move to another setup, you have several options: -3) Create an empty, descriptor-enabled, watch-only wallet in Bitcoin Core: +### Copy the bitcoin core wallet and `eclair-signer.conf` + +Copy your wallet file to a new Bitcoin Core node, and load it with `bitcoin-cli loadwallet`. Bitcoin Core may need to scan the blockchain which could take some time. +Copy `eclair-signer.conf` to your Eclair data directory and set `eclair.bitcoind.wallet` to the name of the wallet configured in `eclair-signer.conf` (and which should also be the name of your Bitcoin Core wallet). +Once your wallet has been imported, just restart Eclair. + +### Create an empty wallet manually and import Eclair descriptors + +1) Create an empty, descriptor-enabled, watch-only wallet in Bitcoin Core: :warning: The name must match the one that you set in `eclair-signer.conf` (here we use "eclair") ```shell $ bitcoin-cli -named createwallet wallet_name=eclair disable_private_keys=true blank=true descriptors=true load_on_startup=true ``` -4) Import public descriptors generated by Eclair +2) Import public descriptors generated by Eclair -`eclair-cli listdescriptors` will return public wallet descriptors in a format that is compatible with Bitcoin Core, and that you can import with `bitcoin-cli -rpcwallet=eclair importdescriptors` -For now, this descriptors follow the BIP84 standard (p2wpkh outputs). +Copy `eclair-signer.conf` to your Eclair data directory but do not change `eclair.bitcoind.wallet`, and restart Eclair. + +`eclair-cli getdescriptors` will return public wallet descriptors in a format that is compatible with Bitcoin Core, and that you can import with `bitcoin-cli -rpcwallet=eclair importdescriptors` This is an example of descriptors generated by Eclair: ```json [ - { - "desc": "wpkh([0d9250da/84h/1h/0h]tpubDDGF9PnrXww2h1mKNjKiXoqdDFGEcZGCZUNq7g26LdzKXKiE31RrFWsogPy1uMLrbG8ksQ8eJS6u6KFLjYUUSVJRuwmMD2SYCr8uG1TcRgM/0/*)#jz5n2pcp", - "internal": false, - "timestamp": 1686055705, - "active": true - }, - { - "desc": "wpkh([0d9250da/84h/1h/0h]tpubDDGF9PnrXww2h1mKNjKiXoqdDFGEcZGCZUNq7g26LdzKXKiE31RrFWsogPy1uMLrbG8ksQ8eJS6u6KFLjYUUSVJRuwmMD2SYCr8uG1TcRgM/1/*)#rk3jh5ge", - "internal": true, - "timestamp": 1686055705, - "active": true - } + { + "desc": "wpkh([0d9250da/84h/1h/0h]tpubDDGF9PnrXww2h1mKNjKiXoqdDFGEcZGCZUNq7g26LdzKXKiE31RrFWsogPy1uMLrbG8ksQ8eJS6u6KFLjYUUSVJRuwmMD2SYCr8uG1TcRgM/0/*)#jz5n2pcp", + "internal": false, + "timestamp": 1686055705, + "active": true + }, + { + "desc": "wpkh([0d9250da/84h/1h/0h]tpubDDGF9PnrXww2h1mKNjKiXoqdDFGEcZGCZUNq7g26LdzKXKiE31RrFWsogPy1uMLrbG8ksQ8eJS6u6KFLjYUUSVJRuwmMD2SYCr8uG1TcRgM/1/*)#rk3jh5ge", + "internal": true, + "timestamp": 1686055705, + "active": true + } ] ``` -You can combine the generation and import of descriptors with: +You can generate the descriptors with your Eclair node and import them into a Bitcoin node with the following commands: ```shell -$ eclair-cli getdescriptors | jq --raw-output -c | xargs -0 bitcoin-cli -rpcwallet=eclair importdescriptors +$ eclair-cli getdescriptors | jq --raw-output -c > descriptors.json +$ cat descriptors.json | xargs -0 bitcoin-cli -rpcwallet=eclair importdescriptors ``` -:warning: If you are restoring an existing `eclair-signer.conf` file with a timestamp that is fairly old, importing descriptors can take a long time, and your -Bitcoin Core node will not be usable until it's done - -5) Configure Eclair to handle private keys for this wallet +:warning: Importing descriptors can take a long time, and your Bitcoin Core node will not be usable until it's done -Set `eclair.bitcoind.wallet` to the name of the wallet just created (`eclair` in the example above) and restart Eclair. +3) Configure Eclair to handle private keys for this wallet -You now have a Bitcoin Core watch-only wallet for which only your Eclair node can sign transactions. This Bitcoin Core wallet can -safely be copied to another Bitcoin Core node to monitor your onchain funds. - -You can also use `eclair-cli getmasterxpub` to get a BIP32 extended public key that you can import into any compatible Bitcoin wallet -to create a watch-only wallet (Electrum for example) that you can use to monitor your Bitcoin Core balance. - -:warning: this means that your Bitcoin Core wallet cannot send funds on its on (since it cannot access private keys to sign transactions). -To send funds onchain you must use `eclair-cli sendonchain`. - -:warning: to backup the private keys of this wallet you must either backup your mnemonic code and passphrase, or backup the `eclair-signer.conf` file in your eclair -directory (default is `~/.eclair`) along with your channels and node seed files. - -:warning: You can also initialise a backup onchain wallet with the same mnemonic code and passphrase (a hardware wallet for example), but be warned that using them may interfere with your node's operations (for example you may end up -double-spending funding transactions generated by your node). +Set `eclair.bitcoind.wallet` to the name of the wallet in `eclair-signer.conf` restart Eclair. diff --git a/docs/Configure.md b/docs/Configure.md index c11375718b..3e3626610f 100644 --- a/docs/Configure.md +++ b/docs/Configure.md @@ -4,7 +4,7 @@ * [Configuration file](#configuration-file) * [Changing the data directory](#changing-the-data-directory) - * [Splitting the configuration](#splitting-the-configuration)\ + * [Splitting the configuration](#splitting-the-configuration) * [Options reference](#options-reference) * [Customize features](#customize-features) * [Customize feerate tolerance](#customize-feerate-tolerance) diff --git a/docs/release-notes/eclair-vnext.md b/docs/release-notes/eclair-vnext.md index 91970c24d6..96c965537a 100644 --- a/docs/release-notes/eclair-vnext.md +++ b/docs/release-notes/eclair-vnext.md @@ -25,11 +25,9 @@ This configuration section replaces the previous `eclair.on-chain-fees.target-bl ### Managing Bitcoin Core wallet keys -You can now create Bitcoin Core watch-only wallets and have Eclair manage their private keys: +You can now use Eclair to manage the private keys for on-chain funds monitored by a Bitcoin Core watch-only wallet. -1. Create an empty, descriptors-enabled watch-only wallet in Bitcoin Core -2. Import wallet descriptors generated by Eclair into this wallet -3. Configure Eclair to use this wallet and set the `eclair.bitcoind.use-eclair-signer` to `true` +See `docs/BitcoinCoreKeys.md` for more details. ### API changes diff --git a/eclair-core/src/main/resources/reference.conf b/eclair-core/src/main/resources/reference.conf index ca6e96ea2a..ecf7ae8452 100644 --- a/eclair-core/src/main/resources/reference.conf +++ b/eclair-core/src/main/resources/reference.conf @@ -40,7 +40,6 @@ eclair { // - ignore: eclair will leave these utxos locked and start startup-locked-utxos-behavior = "stop" final-pubkey-refresh-delay = 3 seconds - use-eclair-signer = false } node-alias = "eclair" diff --git a/eclair-core/src/main/scala/fr/acinq/eclair/Eclair.scala b/eclair-core/src/main/scala/fr/acinq/eclair/Eclair.scala index 045c35c4cc..f49d7df1a8 100644 --- a/eclair-core/src/main/scala/fr/acinq/eclair/Eclair.scala +++ b/eclair-core/src/main/scala/fr/acinq/eclair/Eclair.scala @@ -131,7 +131,7 @@ trait Eclair { def sentInfo(id: PaymentIdentifier)(implicit timeout: Timeout): Future[Seq[OutgoingPayment]] - def sendOnChain(address: String, amount: Satoshi, confirmationTarget: Long): Future[ByteVector32] + def sendOnChain(address: String, amount: Satoshi, confirmationTargetOrFeerate: Either[Long, FeeratePerByte]): Future[ByteVector32] def cpfpBumpFees(targetFeeratePerByte: FeeratePerByte, outpoints: Set[OutPoint]): Future[ByteVector32] @@ -357,11 +357,14 @@ class EclairImpl(appKit: Kit) extends Eclair with Logging { } } - override def sendOnChain(address: String, amount: Satoshi, confirmationTarget: Long): Future[ByteVector32] = { - val feeRate = if (confirmationTarget < 3) appKit.nodeParams.currentFeerates.fast - else if (confirmationTarget > 6) appKit.nodeParams.currentFeerates.slow - else appKit.nodeParams.currentFeerates.medium - + override def sendOnChain(address: String, amount: Satoshi, confirmationTargetOrFeerate: Either[Long, FeeratePerByte]): Future[ByteVector32] = { + val feeRate = confirmationTargetOrFeerate match { + case Left(blocks) => + if (blocks < 3) appKit.nodeParams.currentFeerates.fast + else if (blocks > 6) appKit.nodeParams.currentFeerates.slow + else appKit.nodeParams.currentFeerates.medium + case Right(feeratePerByte) => FeeratePerKw(feeratePerByte) + } appKit.wallet match { case w: BitcoinCoreClient => addressToPublicKeyScript(appKit.nodeParams.chainHash, address) match { diff --git a/eclair-core/src/main/scala/fr/acinq/eclair/Setup.scala b/eclair-core/src/main/scala/fr/acinq/eclair/Setup.scala index e0114e403b..a22ffe643f 100644 --- a/eclair-core/src/main/scala/fr/acinq/eclair/Setup.scala +++ b/eclair-core/src/main/scala/fr/acinq/eclair/Setup.scala @@ -75,8 +75,7 @@ import scala.util.{Failure, Success} class Setup(val datadir: File, pluginParams: Seq[PluginParams], seeds_opt: Option[Seeds] = None, - db: Option[Databases] = None, - onchainKeyManager_opt: Option[LocalOnchainKeyManager] = None)(implicit system: ActorSystem) extends Logging { + db: Option[Databases] = None)(implicit system: ActorSystem) extends Logging { implicit val timeout: Timeout = Timeout(30 seconds) implicit val formats: org.json4s.Formats = org.json4s.DefaultFormats @@ -122,6 +121,8 @@ class Setup(val datadir: File, // early checks PortChecker.checkAvailable(serverBindingAddress) + val onchainKeyManager_opt = LocalOnchainKeyManager.load(datadir, NodeParams.hashFromChain(chain)) + val (bitcoin, bitcoinChainHash) = { val wallet = { val name = config.getString("bitcoind.wallet") @@ -140,6 +141,19 @@ class Setup(val datadir: File, host = config.getString("bitcoind.host"), port = config.getInt("bitcoind.rpcport"), wallet = wallet) + + def createDescriptorWallet(wallets: List[String]): Future[Boolean] = { + if (wallet.exists(name => wallets.contains(name))) { + // wallet already exists + Future.successful(true) + } else { + new BitcoinCoreClient(bitcoinClient, onchainKeyManager_opt).createDescriptorWallet().recover { case e => + logger.error(s"cannot create descriptor wallet", e) + throw BitcoinWalletNotCreatedException(wallet.getOrElse("")) + } + } + } + val future = for { json <- bitcoinClient.invoke("getblockchaininfo").recover { case e => throw BitcoinRPCConnectionException(e) } // Make sure wallet support is enabled in bitcoind. @@ -147,6 +161,7 @@ class Setup(val datadir: File, .collect { case JArray(values) => values.map(value => value.extract[String]) } + true <- createDescriptorWallet(wallets) progress = (json \ "verificationprogress").extract[Double] ibd = (json \ "initialblockdownload").extract[Boolean] blocks = (json \ "blocks").extract[Long] @@ -247,7 +262,7 @@ class Setup(val datadir: File, finalPubkey = new AtomicReference[PublicKey](null) pubkeyRefreshDelay = FiniteDuration(config.getDuration("bitcoind.final-pubkey-refresh-delay").getSeconds, TimeUnit.SECONDS) - bitcoinClient = new BitcoinCoreClient(bitcoin, onchainKeyManager_opt.orElse(LocalOnchainKeyManager.load(datadir, nodeParams.chainHash))) with OnchainPubkeyCache { + bitcoinClient = new BitcoinCoreClient(bitcoin, onchainKeyManager_opt) with OnchainPubkeyCache { val refresher: typed.ActorRef[OnchainPubkeyRefresher.Command] = system.spawn(Behaviors.supervise(OnchainPubkeyRefresher(this, finalPubkey, pubkeyRefreshDelay)).onFailure(typed.SupervisorStrategy.restart), name = "onchain-address-manager") override def getP2wpkhPubkey(renew: Boolean): PublicKey = { @@ -463,6 +478,8 @@ case class BitcoinDefaultWalletException(loaded: List[String]) extends RuntimeEx case class BitcoinWalletNotLoadedException(wallet: String, loaded: List[String]) extends RuntimeException(s"configured wallet \"$wallet\" not in the set of loaded bitcoind wallets: ${loaded.map("\"" + _ + "\"").mkString("[", ",", "]")}") +case class BitcoinWalletNotCreatedException(wallet: String) extends RuntimeException(s"configured wallet \"$wallet\" does not exist and could not be created.") + case object EmptyAPIPasswordException extends RuntimeException("must set a password for the json-rpc api") case object IncompatibleDBException extends RuntimeException("database is not compatible with this version of eclair") diff --git a/eclair-core/src/main/scala/fr/acinq/eclair/blockchain/bitcoind/rpc/BitcoinCoreClient.scala b/eclair-core/src/main/scala/fr/acinq/eclair/blockchain/bitcoind/rpc/BitcoinCoreClient.scala index 1edd2ebe1b..afc33791ef 100644 --- a/eclair-core/src/main/scala/fr/acinq/eclair/blockchain/bitcoind/rpc/BitcoinCoreClient.scala +++ b/eclair-core/src/main/scala/fr/acinq/eclair/blockchain/bitcoind/rpc/BitcoinCoreClient.scala @@ -35,6 +35,7 @@ import org.json4s.JsonAST._ import scodec.bits.ByteVector import java.util.Base64 +import scala.concurrent.duration._ import scala.concurrent.{ExecutionContext, Future} import scala.jdk.CollectionConverters.ListHasAsScala import scala.util.{Failure, Success, Try} @@ -58,6 +59,34 @@ class BitcoinCoreClient(val rpcClient: BitcoinJsonRPCClient, val onchainKeyManag val useEclairSigner = onchainKeyManager_opt.exists(m => rpcClient.wallet.contains(m.wallet)) + //------------------------- WALLET -------------------------// + def importDescriptors(descriptors: Seq[Descriptor])(implicit ec: ExecutionContext): Future[Boolean] = { + rpcClient.invoke("importdescriptors", descriptors).collect { + case JArray(results) => results.forall(item => { + val JBool(success) = item \ "success" + val JArray(_) = item \ "warnings" + success + }) + } + } + + def createDescriptorWallet()(implicit ec: ExecutionContext): Future[Boolean] = { + onchainKeyManager_opt match { + case None => Future.successful(true) // no eclair-backed wallet is configured + case Some(onchainKeyManager) if !rpcClient.wallet.contains(onchainKeyManager.wallet) => Future.successful(true) // configured wallet has a different name + case Some(onchainKeyManager) if !onchainKeyManager.getDescriptors(0).descriptors.forall(desc => TimestampSecond(desc.timestamp) >= TimestampSecond.now() - 2.hours) => + logger.warn(s"descriptors are too old, you will need to manually import them and select how far back to rescan") + Future.failed(new RuntimeException("Could not import descriptors, please check logs for details")) + case Some(onchainKeyManager) => + logger.info(s"Creating a new on-chain eclair-backed wallet in bitcoind: ${onchainKeyManager.wallet}") + for { + _ <- rpcClient.invoke("createwallet", onchainKeyManager.wallet, true, true, "", false, true, true) + _ = logger.info(s"importing new descriptors ${onchainKeyManager.getDescriptors(0).descriptors}") + result <- importDescriptors(onchainKeyManager.getDescriptors(0).descriptors) + } yield result + } + } + //------------------------- TRANSACTIONS -------------------------// def getTransaction(txid: ByteVector32)(implicit ec: ExecutionContext): Future[Transaction] = @@ -796,7 +825,7 @@ object BitcoinCoreClient { def toSatoshi(btcAmount: BigDecimal): Satoshi = Satoshi(btcAmount.bigDecimal.scaleByPowerOfTen(8).longValue) - case class Descriptor(desc: String, internal: Boolean = false, timestamp: Either[String, Long] = Left("now"), active: Boolean = true) + case class Descriptor(desc: String, internal: Boolean = false, timestamp: Long, active: Boolean = true) case class Descriptors(wallet_name: String, descriptors: Seq[Descriptor]) } \ No newline at end of file diff --git a/eclair-core/src/main/scala/fr/acinq/eclair/crypto/keymanager/LocalOnchainKeyManager.scala b/eclair-core/src/main/scala/fr/acinq/eclair/crypto/keymanager/LocalOnchainKeyManager.scala index 405a69cabd..6da6e901a3 100644 --- a/eclair-core/src/main/scala/fr/acinq/eclair/crypto/keymanager/LocalOnchainKeyManager.scala +++ b/eclair-core/src/main/scala/fr/acinq/eclair/crypto/keymanager/LocalOnchainKeyManager.scala @@ -82,8 +82,8 @@ class LocalOnchainKeyManager(override val wallet: String, seed: ByteVector, time val changeDesc = s"wpkh([${this.fingerPrintHex}/$keyPath]${encode(accountPub, prefix)}/1/*)" Descriptors(wallet_name = wallet, descriptors = List( - Descriptor(desc = s"$receiveDesc#${descriptorChecksum(receiveDesc)}", internal = false, active = true, timestamp = Right(timestamp.toLong)), - Descriptor(desc = s"$changeDesc#${descriptorChecksum(changeDesc)}", internal = true, active = true, timestamp = Right(timestamp.toLong)), + Descriptor(desc = s"$receiveDesc#${descriptorChecksum(receiveDesc)}", internal = false, active = true, timestamp = timestamp.toLong), + Descriptor(desc = s"$changeDesc#${descriptorChecksum(changeDesc)}", internal = true, active = true, timestamp = timestamp.toLong), )) } diff --git a/eclair-core/src/test/scala/fr/acinq/eclair/blockchain/bitcoind/BitcoinCoreClientSpec.scala b/eclair-core/src/test/scala/fr/acinq/eclair/blockchain/bitcoind/BitcoinCoreClientSpec.scala index 32b2b9e0f8..0c3283f7dc 100644 --- a/eclair-core/src/test/scala/fr/acinq/eclair/blockchain/bitcoind/BitcoinCoreClientSpec.scala +++ b/eclair-core/src/test/scala/fr/acinq/eclair/blockchain/bitcoind/BitcoinCoreClientSpec.scala @@ -83,7 +83,6 @@ class BitcoinCoreClientSpec extends TestKitBaseClass with BitcoindService with A sender.expectMsgType[JValue] } - test("fund transactions") { val sender = TestProbe() val bitcoinClient = makeBitcoinCoreClient @@ -1441,10 +1440,10 @@ class BitcoinCoreClientWithEclairSignerSpec extends BitcoinCoreClientSpec { val master = DeterministicWallet.generate(seed) val onchainKeyManager = new LocalOnchainKeyManager(s"eclair_$hex", seed, TimestampSecond.now(), Block.RegtestGenesisBlock.hash) - bitcoinrpcclient.invoke("createwallet", s"eclair_$hex", true, false, "", false, true, true, false).pipeTo(sender.ref) - sender.expectMsgType[JValue] val jsonRpcClient = new BasicBitcoinJsonRPCClient(rpcAuthMethod = bitcoinrpcauthmethod, host = "localhost", port = bitcoindRpcPort, wallet = Some(s"eclair_$hex")) - importEclairDescriptors(jsonRpcClient, onchainKeyManager) + val wallet1 = new BitcoinCoreClient(jsonRpcClient, Some(onchainKeyManager)) + wallet1.createDescriptorWallet().pipeTo(sender.ref) + sender.expectMsg(true) // this account xpub can be used to create a watch-only wallet val accountXPub = DeterministicWallet.encode( @@ -1452,7 +1451,6 @@ class BitcoinCoreClientWithEclairSignerSpec extends BitcoinCoreClientSpec { DeterministicWallet.vpub) assert(onchainKeyManager.getOnchainMasterPubKey(0) == accountXPub) - val wallet1 = new BitcoinCoreClient(jsonRpcClient, Some(onchainKeyManager)) def getBip32Path(address: String): DeterministicWallet.KeyPath = { wallet1.rpcClient.invoke("getaddressinfo", address).pipeTo(sender.ref) @@ -1482,18 +1480,15 @@ class BitcoinCoreClientWithEclairSignerSpec extends BitcoinCoreClientSpec { val seed = randomBytes32() val hex = seed.toString() val onchainKeyManager = new LocalOnchainKeyManager(s"eclair_$hex", seed, TimestampSecond.now(), Block.RegtestGenesisBlock.hash) - bitcoinrpcclient.invoke("createwallet", s"eclair_$hex", true, false, "", false, true, true, false).pipeTo(sender.ref) - sender.expectMsgType[JValue] - val jsonRpcClient = new BasicBitcoinJsonRPCClient(rpcAuthMethod = bitcoinrpcauthmethod, host = "localhost", port = bitcoindRpcPort, wallet = Some(s"eclair_$hex")) - importEclairDescriptors(jsonRpcClient, onchainKeyManager) val wallet1 = new BitcoinCoreClient(jsonRpcClient, Some(onchainKeyManager)) + wallet1.createDescriptorWallet().pipeTo(sender.ref) + sender.expectMsg(true) wallet1.getReceiveAddress().pipeTo(sender.ref) val address = sender.expectMsgType[String] // we can send to an onchain address if eclair signs the transactions sendToAddress(address, 100_0000.sat) - generateBlocks(1) // but bitcoin core's sendtoaddress RPC call will fail because wallets uses an external signer diff --git a/eclair-core/src/test/scala/fr/acinq/eclair/blockchain/bitcoind/BitcoindService.scala b/eclair-core/src/test/scala/fr/acinq/eclair/blockchain/bitcoind/BitcoindService.scala index 3dbcbc01e4..39011ab6ff 100644 --- a/eclair-core/src/test/scala/fr/acinq/eclair/blockchain/bitcoind/BitcoindService.scala +++ b/eclair-core/src/test/scala/fr/acinq/eclair/blockchain/bitcoind/BitcoindService.scala @@ -21,17 +21,16 @@ import akka.pattern.pipe import akka.testkit.{TestKitBase, TestProbe} import fr.acinq.bitcoin.psbt.Psbt import fr.acinq.bitcoin.scalacompat.Crypto.PrivateKey -import fr.acinq.bitcoin.scalacompat.{Block, Btc, BtcAmount, MilliBtc, Satoshi, Transaction, TxOut, addressToPublicKeyScript, computeP2WpkhAddress} +import fr.acinq.bitcoin.scalacompat.{Block, Btc, BtcAmount, MilliBtc, MnemonicCode, Satoshi, Transaction, TxOut, addressToPublicKeyScript, computeP2WpkhAddress} import fr.acinq.eclair.blockchain.bitcoind.rpc.BitcoinCoreClient.PreviousTx import fr.acinq.eclair.blockchain.bitcoind.rpc.BitcoinJsonRPCAuthMethod.{SafeCookie, UserPassword} import fr.acinq.eclair.blockchain.bitcoind.rpc.{BasicBitcoinJsonRPCClient, BitcoinCoreClient, BitcoinJsonRPCAuthMethod, BitcoinJsonRPCClient} import fr.acinq.eclair.blockchain.fee.{FeeratePerByte, FeeratePerKB, FeeratePerKw} -import fr.acinq.eclair.crypto.keymanager.{LocalOnchainKeyManager, OnchainKeyManager} +import fr.acinq.eclair.crypto.keymanager.LocalOnchainKeyManager import fr.acinq.eclair.integration.IntegrationSpec import fr.acinq.eclair.{BlockHeight, TestUtils, TimestampSecond, randomKey} import grizzled.slf4j.Logging import org.json4s.JsonAST._ -import scodec.bits.ByteVector import sttp.client3.okhttp.OkHttpFutureBackend import java.io.File @@ -73,7 +72,25 @@ trait BitcoindService extends Logging { var bitcoinrpcclient: BitcoinJsonRPCClient = _ var bitcoinrpcauthmethod: BitcoinJsonRPCAuthMethod = _ var bitcoincli: ActorRef = _ - val onchainKeyManager = new LocalOnchainKeyManager("eclair", ByteVector.fromValidHex("01" * 32), TimestampSecond.now(), Block.RegtestGenesisBlock.hash) + val mnemonics = "legal winner thank year wave sausage worth useful legal winner thank year wave sausage worth useful legal winner thank year wave sausage worth title" + val passphrase = "" + val eclairSignerConf = + s""" + |{ + | eclair { + | signer { + | wallet = "eclair" + | mnemonics = $mnemonics + | passphrase = "$passphrase" + | timestamp = ${TimestampSecond.now().toLong} + | } + | } + |} + |""".stripMargin + val onchainKeyManager = { + new LocalOnchainKeyManager("eclair", MnemonicCode.toSeed(mnemonics, passphrase), TimestampSecond.now(), Block.RegtestGenesisBlock.hash) + } + def startBitcoind(useCookie: Boolean = false, defaultAddressType_opt: Option[String] = None, mempoolSize_opt: Option[Int] = None, // mempool size in MB @@ -157,12 +174,8 @@ trait BitcoindService extends Logging { val sender = TestProbe() waitForBitcoindUp(sender) if (useEclairSigner) { - // wallet_name, disable_private_keys, blank, passphrase, avoid_reuse, descriptors, load_on_startup, external_signer - bitcoinrpcclient.invoke("createwallet", defaultWallet, true, false, "", false, true, true, false).pipeTo(sender.ref) - sender.expectMsgType[JValue] - - val jsonRpcClient = new BasicBitcoinJsonRPCClient(rpcAuthMethod = bitcoinrpcauthmethod, host = "localhost", port = bitcoindRpcPort, wallet = Some(defaultWallet)) - importEclairDescriptors(jsonRpcClient, onchainKeyManager) + makeBitcoinCoreClient.createDescriptorWallet().pipeTo(sender.ref) + sender.expectMsg(true) } else { sender.send(bitcoincli, BitcoinReq("createwallet", defaultWallet)) sender.expectMsgType[JValue] @@ -172,13 +185,6 @@ trait BitcoindService extends Logging { awaitCond(currentBlockHeight(sender) >= BlockHeight(150), max = 3 minutes, interval = 2 second) } - - def importEclairDescriptors(jsonRpcClient: BitcoinJsonRPCClient, keyManager: OnchainKeyManager, probe: TestProbe = TestProbe()): Unit = { - val descriptors = keyManager.getDescriptors(0).descriptors - jsonRpcClient.invoke("importdescriptors", descriptors).pipeTo(probe.ref) - probe.expectMsgType[JValue] - } - def generateBlocks(blockCount: Int, address: Option[String] = None, timeout: FiniteDuration = 10 seconds)(implicit system: ActorSystem): Unit = { val sender = TestProbe() val addressToUse = address match { diff --git a/eclair-core/src/test/scala/fr/acinq/eclair/channel/publish/ReplaceableTxPublisherSpec.scala b/eclair-core/src/test/scala/fr/acinq/eclair/channel/publish/ReplaceableTxPublisherSpec.scala index a9fc08575e..adbfbaadd1 100644 --- a/eclair-core/src/test/scala/fr/acinq/eclair/channel/publish/ReplaceableTxPublisherSpec.scala +++ b/eclair-core/src/test/scala/fr/acinq/eclair/channel/publish/ReplaceableTxPublisherSpec.scala @@ -41,7 +41,6 @@ import fr.acinq.eclair.transactions.Transactions import fr.acinq.eclair.transactions.Transactions._ import fr.acinq.eclair.wire.protocol.{CommitSig, RevokeAndAck} import fr.acinq.eclair.{BlockHeight, MilliSatoshi, MilliSatoshiLong, NodeParams, NotificationsLogger, TestConstants, TestKitBaseClass, TimestampSecond, randomKey} -import org.json4s.JValue import org.scalatest.BeforeAndAfterAll import org.scalatest.funsuite.AnyFunSuiteLike import scodec.bits.ByteVector @@ -1657,20 +1656,18 @@ class ReplaceableTxPublisherWithEclairSignerSpec extends ReplaceableTxPublisherS val entropy = ByteVector.fromValidHex("01" * 32) val seed = MnemonicCode.toSeed(MnemonicCode.toMnemonics(entropy), walletName) val keyManager = new LocalOnchainKeyManager(walletName, seed, TimestampSecond.now(), Block.RegtestGenesisBlock.hash) - bitcoinrpcclient.invoke("createwallet", walletName, true, false, "", false, true, true, false).pipeTo(probe.ref) - probe.expectMsgType[JValue] - val walletRpcClient = new BasicBitcoinJsonRPCClient(rpcAuthMethod = bitcoinrpcauthmethod, host = "localhost", port = bitcoindRpcPort, wallet = Some(walletName)) - importEclairDescriptors(walletRpcClient, keyManager) val walletClient = new BitcoinCoreClient(walletRpcClient, Some(keyManager)) with OnchainPubkeyCache { - val pubkey = { + lazy val pubkey = { getP2wpkhPubkey().pipeTo(probe.ref) probe.expectMsgType[PublicKey] } override def getP2wpkhPubkey(renew: Boolean): PublicKey = pubkey } - + walletClient.createDescriptorWallet().pipeTo(probe.ref) + probe.expectMsg(true) + (walletRpcClient, walletClient) } } \ No newline at end of file diff --git a/eclair-core/src/test/scala/fr/acinq/eclair/integration/IntegrationSpec.scala b/eclair-core/src/test/scala/fr/acinq/eclair/integration/IntegrationSpec.scala index dd6baf7413..a4f6a3bf1a 100644 --- a/eclair-core/src/test/scala/fr/acinq/eclair/integration/IntegrationSpec.scala +++ b/eclair-core/src/test/scala/fr/acinq/eclair/integration/IntegrationSpec.scala @@ -35,6 +35,7 @@ import org.scalatest.BeforeAndAfterAll import org.scalatest.funsuite.AnyFunSuiteLike import java.io.File +import java.nio.file.Files import java.util.Properties import scala.concurrent.Await import scala.concurrent.duration._ @@ -146,8 +147,12 @@ abstract class IntegrationSpec extends TestKitBaseClass with BitcoindService wit def instantiateEclairNode(name: String, config: Config): Unit = { val datadir = new File(INTEGRATION_TMP_DIR, s"datadir-eclair-$name") datadir.mkdirs() + if (useEclairSigner) { + Files.writeString(datadir.toPath.resolve("eclair-signer.conf"), eclairSignerConf) + } implicit val system: ActorSystem = ActorSystem(s"system-$name", config) - val setup = new Setup(datadir, pluginParams = Seq.empty, seeds_opt = Some(Setup.Seeds(randomBytes32(), randomBytes32())), onchainKeyManager_opt = Some(onchainKeyManager)) + + val setup = new Setup(datadir, pluginParams = Seq.empty, seeds_opt = Some(Setup.Seeds(randomBytes32(), randomBytes32()))) val kit = Await.result(setup.bootstrap, 10 seconds) nodes = nodes + (name -> kit) } diff --git a/eclair-node/src/main/scala/fr/acinq/eclair/api/handlers/OnChain.scala b/eclair-node/src/main/scala/fr/acinq/eclair/api/handlers/OnChain.scala index 9ed4366d05..e89b94c7ce 100644 --- a/eclair-node/src/main/scala/fr/acinq/eclair/api/handlers/OnChain.scala +++ b/eclair-node/src/main/scala/fr/acinq/eclair/api/handlers/OnChain.scala @@ -34,9 +34,15 @@ trait OnChain { } val sendOnChain: Route = postRequest("sendonchain") { implicit t => - formFields("address".as[String], "amountSatoshis".as[Satoshi], "confirmationTarget".as[Long]) { - (address, amount, confirmationTarget) => - complete(eclairApi.sendOnChain(address, amount, confirmationTarget)) + formFields("address".as[String], "amountSatoshis".as[Satoshi], "confirmationTarget".as[Long].?, "feeRatePerByte".as[Int].?) { + (address, amount, confirmationTarget_opt, feeratePerByte_opt) => { + val confirmationTargetOrFeerate = (feeratePerByte_opt, confirmationTarget_opt) match { + case (Some(feeratePerByte), _) => Right(FeeratePerByte(Satoshi(feeratePerByte))) + case (None, Some(confirmationTarget)) => Left(confirmationTarget) + case _ => throw new IllegalArgumentException("You must provide a confirmation target (in blocks) or a fee rate (in sat/vb)") + } + complete(eclairApi.sendOnChain(address, amount, confirmationTargetOrFeerate)) + } } } @@ -66,8 +72,8 @@ trait OnChain { } val getmasterxpub: Route = postRequest("getmasterxpub") { implicit t => - formFields("account".as[Long]) { account => - val xpub = this.eclairApi.getOnchainMasterPubKey(account) + formFields("account".as[Long].?) { account_opt => + val xpub = this.eclairApi.getOnchainMasterPubKey(account_opt.getOrElse(0L)) complete(new JObject(List("xpub" -> JString(xpub)))) } } From e28c3f31c5068501522af44d2c4e5ad7e62d2657 Mon Sep 17 00:00:00 2001 From: sstone Date: Tue, 1 Aug 2023 10:06:36 +0200 Subject: [PATCH 3/8] Address review comments * Fix typos, use more explicit method names, ... * Use hardcoded ports in tests instead of "first available port" * User-friendly error message when eclair-baked wallet creation fails at startup * Simplify ReplaceableTxFunder * Replace EitherKt with Scala's Either type * Update signPsbt() to return Try * Skip validation of local non-change outputs: Local non-change outputs send to an external address, for splicing out funds for example. We assume that these inputs are trusted i.e have been created by a trusted API call and our local onchain key manager will not validate them during the signing process. * Do not track our wallet inputs/outputs It is currently easy to identify the wallet inputs/outputs that we added to fund/bump transactions, we don't need to explicitly track them. * Document why we use a separate, specific file for the onchain key manager Using a new signer section is eclair.conf would be simpler but "leaks" because it becomes available everywhere in the code through the actor system's settings instead of being contained to where it is actually needed and could potentially be exposed through a bug that "exports" the configuration (through logs, ....) though this is highly unlikely. --- docs/BitcoinCoreKeys.md | 8 +- .../main/scala/fr/acinq/eclair/Setup.scala | 7 +- .../bitcoind/rpc/BitcoinCoreClient.scala | 15 ++- .../channel/fund/InteractiveTxBuilder.scala | 11 +- .../channel/publish/ReplaceableTxFunder.scala | 121 ++++++++++-------- .../publish/ReplaceableTxPrePublisher.scala | 27 ++-- .../keymanager/LocalOnchainKeyManager.scala | 31 +++-- .../crypto/keymanager/OnchainKeyManager.scala | 11 +- .../bitcoind/BitcoinCoreClientSpec.scala | 44 +++---- .../blockchain/bitcoind/BitcoindService.scala | 2 +- .../publish/ReplaceableTxFunderSpec.scala | 6 +- .../publish/ReplaceableTxPublisherSpec.scala | 2 +- .../LocalOnchainKeyManagerSpec.scala | 66 ++++------ .../integration/ChannelIntegrationSpec.scala | 11 +- .../eclair/integration/IntegrationSpec.scala | 3 +- .../integration/PaymentIntegrationSpec.scala | 2 +- .../AnnouncementsBatchValidationSpec.scala | 7 +- 17 files changed, 196 insertions(+), 178 deletions(-) diff --git a/docs/BitcoinCoreKeys.md b/docs/BitcoinCoreKeys.md index 4f9eeb2617..b01f5c5b64 100644 --- a/docs/BitcoinCoreKeys.md +++ b/docs/BitcoinCoreKeys.md @@ -1,6 +1,6 @@ # Using Eclair to manage your Bitcoin Core wallet's private keys -You can configure Eclair to control (and never expose) the private keys of your Bitcoin Core wallet. This feature was designed to take advantage of deployements where your Eclair node runs in a +You can configure Eclair to control (and never expose) the private keys of your Bitcoin Core wallet. This feature was designed to take advantage of deployment where your Eclair node runs in a "trusted" runtime environment, but is also very useful if your Bitcoin and Eclair nodes run on different machines for example, with a setup for the Bitcoin host that is less secure than for Eclair (because it is shared among several services for example). @@ -43,7 +43,7 @@ This is an example of `eclair-signer.conf` configuration file: Set `eclair.bitcoind.wallet` to the name of the wallet in your `eclair-signer.conf` file (`eclair` in the example above) and restart Eclair. Eclair will automatically create a new, empty, descriptor-enabled, watch-only wallet in Bitcoin Core and import its descriptors. -:warning: Eclair will not import descriptors if the timestamp set in your `eclair-signer.conf` is more than 2 hours old. If the mnmeonics and +:warning: Eclair will not import descriptors if the timestamp set in your `eclair-signer.conf` is more than 2 hours old. If the mnemonics and passphrase that your are using are new, you can safely update this timestamp, but if they have been used before you will need to follow the steps described in the next section. @@ -59,7 +59,7 @@ To send funds onchain you must use `eclair-cli sendonchain`. :warning: to backup the private keys of this wallet you must either backup your mnemonic code and passphrase, or backup the `eclair-signer.conf` file in your eclair directory (default is `~/.eclair`) along with your channels and node seed files. -:warning: You can also initialise a backup onchain wallet with the same mnemonic code and passphrase (on a hardware wallet for example), but be warned that using them may interfere with your node's operations (for example you may end up +:warning: You can also initialize a backup onchain wallet with the same mnemonic code and passphrase (on a hardware wallet for example), but be warned that using them may interfere with your node's operations (for example you may end up double-spending funding transactions generated by your node). ## Importing an existing Eclair-backed bitcoin core wallet @@ -117,4 +117,4 @@ $ cat descriptors.json | xargs -0 bitcoin-cli -rpcwallet=eclair importdescriptor 3) Configure Eclair to handle private keys for this wallet -Set `eclair.bitcoind.wallet` to the name of the wallet in `eclair-signer.conf` restart Eclair. +Set `eclair.bitcoind.wallet` to the name of the wallet in `eclair-signer.conf` and restart Eclair. diff --git a/eclair-core/src/main/scala/fr/acinq/eclair/Setup.scala b/eclair-core/src/main/scala/fr/acinq/eclair/Setup.scala index a22ffe643f..f284cfb061 100644 --- a/eclair-core/src/main/scala/fr/acinq/eclair/Setup.scala +++ b/eclair-core/src/main/scala/fr/acinq/eclair/Setup.scala @@ -142,12 +142,12 @@ class Setup(val datadir: File, port = config.getInt("bitcoind.rpcport"), wallet = wallet) - def createDescriptorWallet(wallets: List[String]): Future[Boolean] = { + def createEclairBackedWallet(wallets: List[String]): Future[Boolean] = { if (wallet.exists(name => wallets.contains(name))) { // wallet already exists Future.successful(true) } else { - new BitcoinCoreClient(bitcoinClient, onchainKeyManager_opt).createDescriptorWallet().recover { case e => + new BitcoinCoreClient(bitcoinClient, onchainKeyManager_opt).createEclairBackedWallet().recover { case e => logger.error(s"cannot create descriptor wallet", e) throw BitcoinWalletNotCreatedException(wallet.getOrElse("")) } @@ -161,7 +161,8 @@ class Setup(val datadir: File, .collect { case JArray(values) => values.map(value => value.extract[String]) } - true <- createDescriptorWallet(wallets) + walletCreated <- createEclairBackedWallet(wallets) + _ = assert(walletCreated, "Cannot create eclair-backed wallet, check logs for details") progress = (json \ "verificationprogress").extract[Double] ibd = (json \ "initialblockdownload").extract[Boolean] blocks = (json \ "blocks").extract[Long] diff --git a/eclair-core/src/main/scala/fr/acinq/eclair/blockchain/bitcoind/rpc/BitcoinCoreClient.scala b/eclair-core/src/main/scala/fr/acinq/eclair/blockchain/bitcoind/rpc/BitcoinCoreClient.scala index afc33791ef..6e646d9a9e 100644 --- a/eclair-core/src/main/scala/fr/acinq/eclair/blockchain/bitcoind/rpc/BitcoinCoreClient.scala +++ b/eclair-core/src/main/scala/fr/acinq/eclair/blockchain/bitcoind/rpc/BitcoinCoreClient.scala @@ -48,7 +48,7 @@ import scala.util.{Failure, Success, Try} * The Bitcoin Core client provides some high-level utility methods to interact with Bitcoin Core. * * @param rpcClient bitcoin core JSON rpc client - * @param onchainKeyManager_opt optional onchain key manager. If provided and its wallet name matcher our rpcClient wallet's name, it will be used to sign transactions (it is assumed that bitcoin + * @param onchainKeyManager_opt optional onchain key manager. If provided and its wallet name matches our rpcClient wallet's name, it will be used to sign transactions (it is assumed that bitcoin * core uses a watch-only wallet with descriptors generated by Eclair with this onchain key manager) */ class BitcoinCoreClient(val rpcClient: BitcoinJsonRPCClient, val onchainKeyManager_opt: Option[OnchainKeyManager] = None) extends OnChainWallet with Logging { @@ -64,22 +64,23 @@ class BitcoinCoreClient(val rpcClient: BitcoinJsonRPCClient, val onchainKeyManag rpcClient.invoke("importdescriptors", descriptors).collect { case JArray(results) => results.forall(item => { val JBool(success) = item \ "success" - val JArray(_) = item \ "warnings" success }) } } - def createDescriptorWallet()(implicit ec: ExecutionContext): Future[Boolean] = { + def createEclairBackedWallet()(implicit ec: ExecutionContext): Future[Boolean] = { onchainKeyManager_opt match { case None => Future.successful(true) // no eclair-backed wallet is configured case Some(onchainKeyManager) if !rpcClient.wallet.contains(onchainKeyManager.wallet) => Future.successful(true) // configured wallet has a different name - case Some(onchainKeyManager) if !onchainKeyManager.getDescriptors(0).descriptors.forall(desc => TimestampSecond(desc.timestamp) >= TimestampSecond.now() - 2.hours) => + case Some(onchainKeyManager) if onchainKeyManager.getWalletTimestamp() < (TimestampSecond.now() - 2.hours) => logger.warn(s"descriptors are too old, you will need to manually import them and select how far back to rescan") Future.failed(new RuntimeException("Could not import descriptors, please check logs for details")) case Some(onchainKeyManager) => logger.info(s"Creating a new on-chain eclair-backed wallet in bitcoind: ${onchainKeyManager.wallet}") for { + // arguments to createwallet: wallet_name, disable_private_keys=true, blank=true, passphrase="", avoid_reuse=false, descriptors=true, load_on_startup=true + // we create an empty watch-only wallet, and then import our public key descriptors _ <- rpcClient.invoke("createwallet", onchainKeyManager.wallet, true, true, "", false, true, true) _ = logger.info(s"importing new descriptors ${onchainKeyManager.getDescriptors(0).descriptors}") result <- importDescriptors(onchainKeyManager.getDescriptors(0).descriptors) @@ -352,7 +353,7 @@ class BitcoinCoreClient(val rpcClient: BitcoinJsonRPCClient, val onchainKeyManag actualFees = kmp2scala(signed.psbt.computeFees()) _ = require(actualFees == fees, s"actual funding fees $actualFees do not match bitcoin core fee $fees") fundingTx = kmp2scala(extracted.getRight) - actualFeerate = FeeratePerKw((actualFees * 1000) / fundingTx.weight()) + actualFeerate = Transactions.fee2rate(actualFees, fundingTx.weight()) maxFeerate = requestedFeeRate + requestedFeeRate / 2 _ = require(actualFeerate < maxFeerate, s"actual feerate $actualFeerate is more than 50% above requested fee rate $targetFeerate") _ = logger.debug(s"created funding txid=${extracted.getRight.txid} outputIndex=$outputIndex fee=${fees}") @@ -472,7 +473,7 @@ class BitcoinCoreClient(val rpcClient: BitcoinJsonRPCClient, val onchainKeyManag def sign(psbt: Psbt): Future[ProcessPsbtResponse] = if (useEclairSigner) { val onchainKeyManager = onchainKeyManager_opt.getOrElse(throw new RuntimeException("We are configured to use an eclair signer has not been loaded")) // this should not be possible - Try(onchainKeyManager.signPsbt(psbt, ourInputs, ourOutputs)) match { + onchainKeyManager.signPsbt(psbt, ourInputs, ourOutputs) match { case Success(signedPsbt) => Future.successful(ProcessPsbtResponse(signedPsbt, signedPsbt.extract().isRight)) case Failure(error) => Future.failed(error) } @@ -627,7 +628,7 @@ class BitcoinCoreClient(val rpcClient: BitcoinJsonRPCClient, val onchainKeyManag signedPsbt <- unlockIfFails(fundedTx.tx.txid, lockedOutputs)(signPsbt(new Psbt(fundedTx.tx), fundedTx.tx.txIn.indices, fundedTx.tx.txOut.indices.filterNot(_ == theirOutputPos))) signedTx = signedPsbt.finalTx actualFees = kmp2scala(signedPsbt.psbt.computeFees()) - actualFeerate = FeeratePerKw((actualFees * 1000) / signedTx.weight()) + actualFeerate = Transactions.fee2rate(actualFees, signedTx.weight()) maxFeerate = feeratePerKw + feeratePerKw / 2 _ = require(actualFeerate < maxFeerate, s"actual fee rate $actualFeerate is more than 50% above requested fee rate $feeratePerKw") txid <- publishTransaction(signedTx) diff --git a/eclair-core/src/main/scala/fr/acinq/eclair/channel/fund/InteractiveTxBuilder.scala b/eclair-core/src/main/scala/fr/acinq/eclair/channel/fund/InteractiveTxBuilder.scala index f20be9cc18..8e4d216833 100644 --- a/eclair-core/src/main/scala/fr/acinq/eclair/channel/fund/InteractiveTxBuilder.scala +++ b/eclair-core/src/main/scala/fr/acinq/eclair/channel/fund/InteractiveTxBuilder.scala @@ -775,7 +775,14 @@ private class InteractiveTxBuilder(replyTo: ActorRef[InteractiveTxBuilder.Respon } else { // we only sign our wallet inputs, and check that we can spend our wallet outputs val ourWalletInputs = unsignedTx.localInputs.map(i => tx.txIn.indexWhere(_.outPoint == OutPoint(i.previousTx, i.previousTxOutput.toInt))) - val ourWalletOutputs = unsignedTx.localOutputs.map(o => tx.txOut.indexWhere(output => output.amount == o.amount && output.publicKeyScript == o.pubkeyScript)) + val ourWalletOutputs = unsignedTx.localOutputs.collect { + // README!! there are 2 types of local outputs: + // Change, which go back into our wallet + // NonChange, which go to an external address (typically during a splice-out) + // Here we only keep outputs which are ours i.e go back into our wallet + // And we trust that NonChange outputs are valid. This only works if the entry point for creating such outputs is trusted (for example, a secure API call) + case Output.Local.Change(_, amount, pubkeyScript) => tx.txOut.indexWhere(output => output.amount == amount && output.publicKeyScript == pubkeyScript) + } context.pipeToSelf(wallet.signPsbt(new Psbt(tx), ourWalletInputs, ourWalletOutputs).map { response => val localOutpoints = unsignedTx.localInputs.map(_.outPoint).toSet @@ -786,7 +793,7 @@ private class InteractiveTxBuilder(replyTo: ActorRef[InteractiveTxBuilder.Respon val expectedLocalAmountIn = unsignedTx.localInputs.map(i => i.previousTx.txOut(i.previousTxOutput.toInt).amount).sum require(actualLocalAmountIn == expectedLocalAmountIn, s"local spent amount ${actualLocalAmountIn} does not match what we expect ($expectedLocalAmountIn") val actualLocalAmountOut = ourWalletOutputs.map(i => partiallySignedTx.txOut(i).amount).sum - val expectedLocalAmountOut = unsignedTx.localOutputs.map(_.amount).sum + val expectedLocalAmountOut = unsignedTx.localOutputs.collect { case c: Output.Local.Change => c.amount }.sum require(actualLocalAmountOut == expectedLocalAmountOut, s"local output amount ${actualLocalAmountOut} does not match what we expect ($expectedLocalAmountOut") val sigs = partiallySignedTx.txIn.filter(txIn => localOutpoints.contains(txIn.outPoint)).map(_.witness) PartiallySignedSharedTransaction(unsignedTx, TxSignatures(fundingParams.channelId, partiallySignedTx, sigs, sharedSig_opt)) diff --git a/eclair-core/src/main/scala/fr/acinq/eclair/channel/publish/ReplaceableTxFunder.scala b/eclair-core/src/main/scala/fr/acinq/eclair/channel/publish/ReplaceableTxFunder.scala index edc1a81b58..160fdb2d1b 100644 --- a/eclair-core/src/main/scala/fr/acinq/eclair/channel/publish/ReplaceableTxFunder.scala +++ b/eclair-core/src/main/scala/fr/acinq/eclair/channel/publish/ReplaceableTxFunder.scala @@ -19,9 +19,8 @@ package fr.acinq.eclair.channel.publish import akka.actor.typed.eventstream.EventStream import akka.actor.typed.scaladsl.{ActorContext, Behaviors} import akka.actor.typed.{ActorRef, Behavior} -import fr.acinq.bitcoin.psbt.Psbt +import fr.acinq.bitcoin.psbt.{Psbt, UpdateFailure} import fr.acinq.bitcoin.scalacompat.{ByteVector32, OutPoint, Satoshi, Script, Transaction, TxOut} -import fr.acinq.bitcoin.utils.EitherKt import fr.acinq.eclair.NotificationsLogger.NotifyNodeOperator import fr.acinq.eclair.blockchain.OnChainWallet import fr.acinq.eclair.blockchain.bitcoind.rpc.BitcoinCoreClient @@ -52,7 +51,7 @@ object ReplaceableTxFunder { sealed trait Command case class FundTransaction(replyTo: ActorRef[FundingResult], cmd: TxPublisher.PublishReplaceableTx, tx: Either[FundedTx, ReplaceableTxWithWitnessData], targetFeerate: FeeratePerKw) extends Command - private case class AddInputsOk(tx: ReplaceableTxWithWitnessData, totalAmountIn: Satoshi, packageWeight: Int) extends Command + private case class AddInputsOk(tx: ReplaceableTxWithWitnessData, totalAmountIn: Satoshi) extends Command private case class AddInputsFailed(reason: Throwable) extends Command private case class SignWalletInputsOk(signedTx: Transaction) extends Command private case class SignWalletInputsFailed(reason: Throwable) extends Command @@ -254,7 +253,7 @@ private class ReplaceableTxFunder(nodeParams: NodeParams, val htlcFeerate = cmd.commitment.localCommit.spec.htlcTxFeerate(cmd.commitment.params.commitmentFormat) if (targetFeerate <= htlcFeerate) { log.info("publishing {} without adding inputs: txid={}", cmd.desc, htlcTx.txInfo.tx.txid) - sign(txWithWitnessData, htlcFeerate, htlcTx.txInfo.amountIn, htlcTx.txInfo.tx.weight()) + sign(txWithWitnessData, htlcFeerate, htlcTx.txInfo.amountIn) } else { addWalletInputs(htlcTx, targetFeerate) } @@ -266,7 +265,7 @@ private class ReplaceableTxFunder(nodeParams: NodeParams, replyTo ! FundingFailed(TxPublisher.TxRejectedReason.TxSkipped(retryNextBlock = true)) Behaviors.stopped case Right(updatedClaimHtlcTx) => - sign(updatedClaimHtlcTx, targetFeerate, updatedClaimHtlcTx.txInfo.amountIn, updatedClaimHtlcTx.txInfo.tx.weight()) + sign(updatedClaimHtlcTx, targetFeerate, updatedClaimHtlcTx.txInfo.amountIn) } } } @@ -280,7 +279,7 @@ private class ReplaceableTxFunder(nodeParams: NodeParams, Behaviors.stopped case AdjustPreviousTxOutputResult.TxOutputAdjusted(updatedTx) => log.debug("bumping {} fees without adding new inputs: txid={}", cmd.desc, updatedTx.txInfo.tx.txid) - sign(updatedTx, targetFeerate, previousTx.totalAmountIn, updatedTx.txInfo.tx.weight()) + sign(updatedTx, targetFeerate, previousTx.totalAmountIn) case AdjustPreviousTxOutputResult.AddWalletInputs(tx) => log.debug("bumping {} fees requires adding new inputs (feerate={})", cmd.desc, targetFeerate) // We restore the original transaction (remove previous attempt's wallet inputs). @@ -291,13 +290,13 @@ private class ReplaceableTxFunder(nodeParams: NodeParams, private def addWalletInputs(txWithWitnessData: ReplaceableTxWithWalletInputs, targetFeerate: FeeratePerKw): Behavior[Command] = { context.pipeToSelf(addInputs(txWithWitnessData, targetFeerate, cmd.commitment)) { - case Success((fundedTx, totalAmountIn, packageWeight)) => AddInputsOk(fundedTx, totalAmountIn, packageWeight) + case Success((fundedTx, totalAmountIn)) => AddInputsOk(fundedTx, totalAmountIn) case Failure(reason) => AddInputsFailed(reason) } Behaviors.receiveMessagePartial { - case AddInputsOk(fundedTx, totalAmountIn, packageWeight) => + case AddInputsOk(fundedTx, totalAmountIn) => log.info("added {} wallet input(s) and {} wallet output(s) to {}", fundedTx.txInfo.tx.txIn.length - 1, fundedTx.txInfo.tx.txOut.length - 1, cmd.desc) - sign(fundedTx, targetFeerate, totalAmountIn, packageWeight) + sign(fundedTx, targetFeerate, totalAmountIn) case AddInputsFailed(reason) => if (reason.getMessage.contains("Insufficient funds")) { val nodeOperatorMessage = @@ -315,13 +314,13 @@ private class ReplaceableTxFunder(nodeParams: NodeParams, } } - private def sign(fundedTx: ReplaceableTxWithWitnessData, txFeerate: FeeratePerKw, amountIn: Satoshi, packageWeight: Int): Behavior[Command] = { + private def sign(fundedTx: ReplaceableTxWithWitnessData, txFeerate: FeeratePerKw, amountIn: Satoshi): Behavior[Command] = { val channelKeyPath = keyManager.keyPath(cmd.commitment.localParams, cmd.commitment.params.channelConfig) fundedTx match { case claimAnchorTx: ClaimLocalAnchorWithWitnessData => val localSig = keyManager.sign(claimAnchorTx.txInfo, keyManager.fundingPublicKey(cmd.commitment.localParams.fundingKeyPath, cmd.commitment.fundingTxIndex), TxOwner.Local, cmd.commitment.params.commitmentFormat) val signedTx = claimAnchorTx.copy(txInfo = addSigs(claimAnchorTx.txInfo, localSig)) - signWalletInputs(signedTx, txFeerate, amountIn, packageWeight) + signWalletInputs(signedTx, txFeerate, amountIn) case htlcTx: HtlcWithWitnessData => val localPerCommitmentPoint = keyManager.commitmentPoint(channelKeyPath, cmd.commitment.localCommit.index) val localHtlcBasepoint = keyManager.htlcPoint(channelKeyPath) @@ -332,7 +331,7 @@ private class ReplaceableTxFunder(nodeParams: NodeParams, } val hasWalletInputs = htlcTx.txInfo.tx.txIn.size > 1 if (hasWalletInputs) { - signWalletInputs(signedTx, txFeerate, amountIn, packageWeight) + signWalletInputs(signedTx, txFeerate, amountIn) } else { replyTo ! TransactionReady(FundedTx(signedTx, amountIn, txFeerate)) Behaviors.stopped @@ -353,43 +352,58 @@ private class ReplaceableTxFunder(nodeParams: NodeParams, } } - private def signWalletInputs(locallySignedTx: ReplaceableTxWithWalletInputs, txFeerate: FeeratePerKw, amountIn: Satoshi, packageWeight: Int): Behavior[Command] = { + private def signWalletInputs(locallySignedTx: ReplaceableTxWithWalletInputs, txFeerate: FeeratePerKw, amountIn: Satoshi): Behavior[Command] = { import fr.acinq.bitcoin.scalacompat.KotlinUtils._ // we finalize (sign) the input that we control, and will then ask our bitcoin client to sign wallet inputs val psbt = new Psbt(locallySignedTx.txInfo.tx) - val updated = psbt.updateWitnessInput(locallySignedTx.txInfo.input.outPoint, locallySignedTx.txInfo.input.txOut, null, fr.acinq.bitcoin.Script.parse(locallySignedTx.txInfo.input.redeemScript), null, java.util.Map.of()) - val finalized = EitherKt.flatMap(updated, (psbt: Psbt) => psbt.finalizeWitnessInput(0, locallySignedTx.txInfo.tx.txIn.head.witness)) - if (!finalized.isRight) { - log.error(s"cannot sign ${cmd.desc}: ", finalized.getLeft) - unlockAndStop(locallySignedTx.txInfo.input.outPoint, locallySignedTx.txInfo.tx, TxPublisher.TxRejectedReason.UnknownTxFailure) - } else { - val psbt1 = finalized.getRight - val ourWalletInputs = locallySignedTx.walletInputs - val ourWalletOutputs = locallySignedTx.walletOutputs - context.pipeToSelf(bitcoinClient.signPsbt(psbt1, ourWalletInputs, ourWalletOutputs)) { - case Success(processPsbtResponse) => - val signedTx = processPsbtResponse.finalTx - val actualFees = kmp2scala(processPsbtResponse.psbt.computeFees()) - val actualFeerate = FeeratePerKw((actualFees * 1000) / packageWeight) - if (actualFeerate >= txFeerate * 2) { - SignWalletInputsFailed(new RuntimeException(s"actual fee rate $actualFeerate is more than twice the requested fee rate $txFeerate")) - } else { - SignWalletInputsOk(signedTx) - } - case Failure(reason) => SignWalletInputsFailed(reason) - } - Behaviors.receiveMessagePartial { - case SignWalletInputsOk(signedTx) => - val fullySignedTx = locallySignedTx.updateTx(signedTx) - replyTo ! TransactionReady(FundedTx(fullySignedTx, amountIn, txFeerate)) - Behaviors.stopped - case SignWalletInputsFailed(reason) => - log.error(s"cannot sign ${cmd.desc}: ", reason) - // We reply with the failure only once the utxos are unlocked, otherwise there is a risk that our parent stops - // itself, which will automatically stop us before we had a chance to unlock them. - unlockAndStop(locallySignedTx.txInfo.input.outPoint, locallySignedTx.txInfo.tx, TxPublisher.TxRejectedReason.UnknownTxFailure) - } + val updated: Either[UpdateFailure, Psbt] = psbt.updateWitnessInput(locallySignedTx.txInfo.input.outPoint, locallySignedTx.txInfo.input.txOut, null, fr.acinq.bitcoin.Script.parse(locallySignedTx.txInfo.input.redeemScript), null, java.util.Map.of()) + val finalized = updated.flatMap(_.finalizeWitnessInput(0, locallySignedTx.txInfo.tx.txIn.head.witness)) + finalized match { + case Left(failure) => + log.error(s"cannot sign ${cmd.desc}: ", failure) + unlockAndStop(locallySignedTx.txInfo.input.outPoint, locallySignedTx.txInfo.tx, TxPublisher.TxRejectedReason.UnknownTxFailure) + case Right(psbt1) => + // the transaction that we want to fund/replace has one input, the first one. Additional inputs are provided by our onchain wallet. + val ourWalletInputs = locallySignedTx.txInfo.tx.txIn.indices.drop(1) + // for "claim anchor txs" there is a single change output that sends to our onchain wallet + // for htlc txs the first output is the one we want to fund/bump, additional outputs send to our onchain wallet + val ourWalletOutputs = locallySignedTx match { + case _: ClaimLocalAnchorWithWitnessData => Seq(0) + case _: HtlcWithWitnessData => locallySignedTx.txInfo.tx.txOut.indices.drop(1) + } + context.pipeToSelf(bitcoinClient.signPsbt(psbt1, ourWalletInputs, ourWalletOutputs)) { + case Success(processPsbtResponse) => + val signedTx = processPsbtResponse.finalTx + val actualFees = kmp2scala(processPsbtResponse.psbt.computeFees()) + val actualWeight = locallySignedTx match { + case _: ClaimLocalAnchorWithWitnessData => signedTx.weight() + dummySignedCommitTx(cmd.commitment).tx.weight() + case _ => + locallySignedTx.txInfo match { + case _: HtlcSuccessTx => cmd.commitment.params.commitmentFormat.htlcSuccessInputWeight + signedTx.weight() + case _: HtlcTimeoutTx => cmd.commitment.params.commitmentFormat.htlcTimeoutInputWeight + signedTx.weight() + case _ => signedTx.weight() + } + } + val actualFeerate = Transactions.fee2rate(actualFees, actualWeight) + if (actualFeerate >= txFeerate * 2) { + SignWalletInputsFailed(new RuntimeException(s"actual fee rate $actualFeerate is more than twice the requested fee rate $txFeerate")) + } else { + SignWalletInputsOk(signedTx) + } + case Failure(reason) => SignWalletInputsFailed(reason) + } + Behaviors.receiveMessagePartial { + case SignWalletInputsOk(signedTx) => + val fullySignedTx = locallySignedTx.updateTx(signedTx) + replyTo ! TransactionReady(FundedTx(fullySignedTx, amountIn, txFeerate)) + Behaviors.stopped + case SignWalletInputsFailed(reason) => + log.error(s"cannot sign ${cmd.desc}: ", reason) + // We reply with the failure only once the utxos are unlocked, otherwise there is a risk that our parent stops + // itself, which will automatically stop us before we had a chance to unlock them. + unlockAndStop(locallySignedTx.txInfo.input.outPoint, locallySignedTx.txInfo.tx, TxPublisher.TxRejectedReason.UnknownTxFailure) + } } } @@ -405,14 +419,14 @@ private class ReplaceableTxFunder(nodeParams: NodeParams, } } - private def addInputs(tx: ReplaceableTxWithWalletInputs, targetFeerate: FeeratePerKw, commitment: FullCommitment): Future[(ReplaceableTxWithWalletInputs, Satoshi, Int)] = { + private def addInputs(tx: ReplaceableTxWithWalletInputs, targetFeerate: FeeratePerKw, commitment: FullCommitment): Future[(ReplaceableTxWithWalletInputs, Satoshi)] = { tx match { case anchorTx: ClaimLocalAnchorWithWitnessData => addInputs(anchorTx, targetFeerate, commitment) case htlcTx: HtlcWithWitnessData => addInputs(htlcTx, targetFeerate, commitment) } } - private def addInputs(anchorTx: ClaimLocalAnchorWithWitnessData, targetFeerate: FeeratePerKw, commitment: FullCommitment): Future[(ClaimLocalAnchorWithWitnessData, Satoshi, Int)] = { + private def addInputs(anchorTx: ClaimLocalAnchorWithWitnessData, targetFeerate: FeeratePerKw, commitment: FullCommitment): Future[(ClaimLocalAnchorWithWitnessData, Satoshi)] = { import fr.acinq.bitcoin.scalacompat.KotlinUtils._ val dustLimit = commitment.localParams.dustLimit @@ -452,21 +466,20 @@ private class ReplaceableTxFunder(nodeParams: NodeParams, partiallySignedTx = processPsbtResponse.extractPartiallySignedTx dummySignedTx = addSigs(anchorTx.updateTx(partiallySignedTx).txInfo, PlaceHolderSig) packageWeight = commitTx.weight() + dummySignedTx.tx.weight() - // above, we asked bitcoin core to use the package weight to estimate fees when it built and funded this transaction, so we // use the same package weight here to compute the actual fee rate that we get - actualFeerate = FeeratePerKw((processPsbtResponse.psbt.computeFees() * 1000) / packageWeight) + actualFeerate = Transactions.fee2rate(processPsbtResponse.psbt.computeFees(), packageWeight) _ = require(actualFeerate < targetFeerate * 2, s"actual fee rate $actualFeerate is more than twice the requested fee rate $targetFeerate") anchorTxFee = weight2fee(targetFeerate, packageWeight) - weight2fee(commitment.localCommit.spec.commitTxFeerate, commitTx.weight()) changeAmount = dustLimit.max(fundTxResponse.amountIn - anchorTxFee) fundedTx = fundTxResponse.tx.copy(txOut = Seq(changeOutput.copy(amount = changeAmount))) } yield { - (anchorTx.updateTx(fundedTx).updateWalletInputsAndOutputs(ourWalletInputs, ourWalletOutputs), fundTxResponse.amountIn, packageWeight) + (anchorTx.updateTx(fundedTx), fundTxResponse.amountIn) } } - private def addInputs(htlcTx: HtlcWithWitnessData, targetFeerate: FeeratePerKw, commitment: FullCommitment): Future[(HtlcWithWitnessData, Satoshi, Int)] = { + private def addInputs(htlcTx: HtlcWithWitnessData, targetFeerate: FeeratePerKw, commitment: FullCommitment): Future[(HtlcWithWitnessData, Satoshi)] = { import fr.acinq.bitcoin.scalacompat.KotlinUtils._ val htlcInputWeight = InputWeight(htlcTx.txInfo.input.outPoint, htlcTx.txInfo match { @@ -476,16 +489,16 @@ private class ReplaceableTxFunder(nodeParams: NodeParams, bitcoinClient.fundTransaction(htlcTx.txInfo.tx, FundTransactionOptions(targetFeerate, changePosition = Some(1), inputWeights = Seq(htlcInputWeight))).flatMap(fundTxResponse => { val ourWalletInputs = fundTxResponse.tx.txIn.indices.filterNot(i => fundTxResponse.tx.txIn(i).outPoint == htlcTx.txInfo.input.outPoint) val ourWalletOutputs = if (fundTxResponse.tx.txOut.size > 1) Seq(1) else Nil // there may not be a change output - val unsignedTx = htlcTx.updateTx(fundTxResponse.tx).updateWalletInputsAndOutputs(ourWalletInputs, ourWalletOutputs) + val unsignedTx = htlcTx.updateTx(fundTxResponse.tx) val psbt = new Psbt(fundTxResponse.tx) bitcoinClient.signPsbt(psbt, ourWalletInputs, ourWalletOutputs).map(processPsbtResponse => { val actualFees: Satoshi = processPsbtResponse.psbt.computeFees() require(actualFees == fundTxResponse.fee, s"Bitcoin Core fees (${fundTxResponse.fee} do not match ours ($actualFees)") val packageWeight = fundTxResponse.tx.weight() + htlcInputWeight.weight - val actualFeerate = FeeratePerKw((fundTxResponse.fee * 1000) / packageWeight) + val actualFeerate = Transactions.fee2rate(fundTxResponse.fee, packageWeight.toInt) require(actualFeerate < targetFeerate * 2, s"actual fee rate $actualFeerate is more than twice the requested fee rate $targetFeerate") - (unsignedTx, fundTxResponse.amountIn, packageWeight.toInt) + (unsignedTx, fundTxResponse.amountIn) }) }) } diff --git a/eclair-core/src/main/scala/fr/acinq/eclair/channel/publish/ReplaceableTxPrePublisher.scala b/eclair-core/src/main/scala/fr/acinq/eclair/channel/publish/ReplaceableTxPrePublisher.scala index 7ff2166b7c..8990627cf2 100644 --- a/eclair-core/src/main/scala/fr/acinq/eclair/channel/publish/ReplaceableTxPrePublisher.scala +++ b/eclair-core/src/main/scala/fr/acinq/eclair/channel/publish/ReplaceableTxPrePublisher.scala @@ -64,33 +64,22 @@ object ReplaceableTxPrePublisher { def txInfo: ReplaceableTransactionWithInputInfo def updateTx(tx: Transaction): ReplaceableTxWithWitnessData } - /** Replaceable transaction for which we may need to add wallet inputs and outputs. */ + /** Replaceable transaction for which we may need to add wallet inputs. */ sealed trait ReplaceableTxWithWalletInputs extends ReplaceableTxWithWitnessData { override def updateTx(tx: Transaction): ReplaceableTxWithWalletInputs - - def updateWalletInputsAndOutputs(walletIn: Seq[Int], walletOut: Seq[Int]): ReplaceableTxWithWalletInputs - def walletInputs: Seq[Int] - def walletOutputs: Seq[Int] } - case class ClaimLocalAnchorWithWitnessData(txInfo: ClaimLocalAnchorOutputTx, override val walletInputs: Seq[Int], override val walletOutputs: Seq[Int]) extends ReplaceableTxWithWalletInputs { + case class ClaimLocalAnchorWithWitnessData(txInfo: ClaimLocalAnchorOutputTx) extends ReplaceableTxWithWalletInputs { override def updateTx(tx: Transaction): ClaimLocalAnchorWithWitnessData = this.copy(txInfo = this.txInfo.copy(tx = tx)) - - override def updateWalletInputsAndOutputs(walletIn: Seq[Int], walletOut: Seq[Int]): ClaimLocalAnchorWithWitnessData = this.copy(walletInputs = walletIn, walletOutputs = walletOut) - } sealed trait HtlcWithWitnessData extends ReplaceableTxWithWalletInputs { override def txInfo: HtlcTx override def updateTx(tx: Transaction): HtlcWithWitnessData - - override def updateWalletInputsAndOutputs(walletIn: Seq[Int], walletOut: Seq[Int]): HtlcWithWitnessData } - case class HtlcSuccessWithWitnessData(txInfo: HtlcSuccessTx, remoteSig: ByteVector64, preimage: ByteVector32, override val walletInputs: Seq[Int], override val walletOutputs: Seq[Int]) extends HtlcWithWitnessData { + case class HtlcSuccessWithWitnessData(txInfo: HtlcSuccessTx, remoteSig: ByteVector64, preimage: ByteVector32) extends HtlcWithWitnessData { override def updateTx(tx: Transaction): HtlcSuccessWithWitnessData = this.copy(txInfo = this.txInfo.copy(tx = tx)) - override def updateWalletInputsAndOutputs(walletIn: Seq[Int], walletOut: Seq[Int]): HtlcSuccessWithWitnessData = this.copy(walletInputs = walletIn, walletOutputs = walletOut) } - case class HtlcTimeoutWithWitnessData(txInfo: HtlcTimeoutTx, remoteSig: ByteVector64, override val walletInputs: Seq[Int], override val walletOutputs: Seq[Int]) extends HtlcWithWitnessData { + case class HtlcTimeoutWithWitnessData(txInfo: HtlcTimeoutTx, remoteSig: ByteVector64) extends HtlcWithWitnessData { override def updateTx(tx: Transaction): HtlcTimeoutWithWitnessData = this.copy(txInfo = this.txInfo.copy(tx = tx)) - override def updateWalletInputsAndOutputs(walletIn: Seq[Int], walletOut: Seq[Int]): HtlcTimeoutWithWitnessData = this.copy(walletInputs = walletIn, walletOutputs = walletOut) } sealed trait ClaimHtlcWithWitnessData extends ReplaceableTxWithWitnessData { override def txInfo: ClaimHtlcTx @@ -164,7 +153,7 @@ private class ReplaceableTxPrePublisher(nodeParams: NodeParams, } Behaviors.receiveMessagePartial { case ParentTxOk => - replyTo ! PreconditionsOk(ClaimLocalAnchorWithWitnessData(localAnchorTx, Nil, Nil)) + replyTo ! PreconditionsOk(ClaimLocalAnchorWithWitnessData(localAnchorTx)) Behaviors.stopped case FundingTxNotFound => log.debug("funding tx could not be found, we don't know yet if we need to claim our anchor") @@ -182,7 +171,7 @@ private class ReplaceableTxPrePublisher(nodeParams: NodeParams, case UnknownFailure(reason) => log.error(s"could not check ${cmd.desc} preconditions, proceeding anyway: ", reason) // If our checks fail, we don't want it to prevent us from trying to publish our commit tx. - replyTo ! PreconditionsOk(ClaimLocalAnchorWithWitnessData(localAnchorTx, Nil, Nil)) + replyTo ! PreconditionsOk(ClaimLocalAnchorWithWitnessData(localAnchorTx)) Behaviors.stopped } } @@ -239,7 +228,7 @@ private class ReplaceableTxPrePublisher(nodeParams: NodeParams, commitment.changes.localChanges.all.collectFirst { case u: UpdateFulfillHtlc if Crypto.sha256(u.paymentPreimage) == tx.paymentHash => u.paymentPreimage } match { - case Some(preimage) => Some(HtlcSuccessWithWitnessData(tx, remoteSig, preimage, Nil, Nil)) + case Some(preimage) => Some(HtlcSuccessWithWitnessData(tx, remoteSig, preimage)) case None => log.error(s"preimage not found for htlcId=${tx.htlcId}, skipping...") None @@ -252,7 +241,7 @@ private class ReplaceableTxPrePublisher(nodeParams: NodeParams, commitment.localCommit.htlcTxsAndRemoteSigs.collectFirst { case HtlcTxAndRemoteSig(HtlcTimeoutTx(input, _, _, _), remoteSig) if input.outPoint == tx.input.outPoint => remoteSig } match { - case Some(remoteSig) => Some(HtlcTimeoutWithWitnessData(tx, remoteSig, Nil, Nil)) + case Some(remoteSig) => Some(HtlcTimeoutWithWitnessData(tx, remoteSig)) case None => log.error(s"remote signature not found for htlcId=${tx.htlcId}, skipping...") None diff --git a/eclair-core/src/main/scala/fr/acinq/eclair/crypto/keymanager/LocalOnchainKeyManager.scala b/eclair-core/src/main/scala/fr/acinq/eclair/crypto/keymanager/LocalOnchainKeyManager.scala index 6da6e901a3..d92fb83e38 100644 --- a/eclair-core/src/main/scala/fr/acinq/eclair/crypto/keymanager/LocalOnchainKeyManager.scala +++ b/eclair-core/src/main/scala/fr/acinq/eclair/crypto/keymanager/LocalOnchainKeyManager.scala @@ -2,10 +2,9 @@ package fr.acinq.eclair.crypto.keymanager import com.typesafe.config.ConfigFactory import fr.acinq.bitcoin.ScriptWitness -import fr.acinq.bitcoin.psbt.{Psbt, SignPsbtResult} +import fr.acinq.bitcoin.psbt.{Psbt, UpdateFailure} import fr.acinq.bitcoin.scalacompat.DeterministicWallet._ import fr.acinq.bitcoin.scalacompat.{Block, ByteVector32, Crypto, DeterministicWallet, MnemonicCode, computeBIP84Address} -import fr.acinq.bitcoin.utils.EitherKt import fr.acinq.eclair.TimestampSecond import fr.acinq.eclair.blockchain.bitcoind.rpc.BitcoinCoreClient.{Descriptor, Descriptors} import grizzled.slf4j.Logging @@ -13,11 +12,21 @@ import scodec.bits.ByteVector import java.io.File import scala.jdk.CollectionConverters.MapHasAsScala +import scala.util.Try object LocalOnchainKeyManager extends Logging { def descriptorChecksum(span: String): String = fr.acinq.bitcoin.Descriptor.checksum(span) + /** + * Load a configuration file and create an onchain key manager + * + * @param datadir eclair data directory + * @param chainHash chain we're on + * @return a LocalOnchainKeyManager instance if a configuration file exists + */ def load(datadir: File, chainHash: ByteVector32): Option[LocalOnchainKeyManager] = { + // we use a specific file instead of adding values to eclair's configuration file because it is available everywhere in the code through + // the actor system's settings and we'd like to restrict access to the onchain wallet seed val file = new File(datadir, "eclair-signer.conf") if (file.exists()) { val config = ConfigFactory.parseFile(file) @@ -68,6 +77,8 @@ class LocalOnchainKeyManager(override val wallet: String, seed: ByteVector, time DeterministicWallet.encode(accountPub, prefix) } + override def getWalletTimestamp(): TimestampSecond = timestamp + override def getDescriptors(account: Long): Descriptors = { val keyPath = s"$rootPath/$account'".replace('\'', 'h') // Bitcoin Core understands both ' and h suffix for hardened derivation, and h is much easier to parse for external tools val prefix: Int = chainHash match { @@ -87,7 +98,7 @@ class LocalOnchainKeyManager(override val wallet: String, seed: ByteVector, time )) } - override def signPsbt(psbt: Psbt, ourInputs: Seq[Int], ourOutputs: Seq[Int]): Psbt = { + override def signPsbt(psbt: Psbt, ourInputs: Seq[Int], ourOutputs: Seq[Int]): Try[Psbt] = Try { import fr.acinq.bitcoin.scalacompat.KotlinUtils._ val spent = ourInputs.map(i => kmp2scala(psbt.getInput(i).getWitnessUtxo.amount)).sum @@ -124,6 +135,7 @@ class LocalOnchainKeyManager(override val wallet: String, seed: ByteVector, time } private def sigbnPsbtInput(psbt: Psbt, pos: Int): Psbt = { + import fr.acinq.bitcoin.scalacompat.KotlinUtils.eitherkmp2either import fr.acinq.bitcoin.{Script, SigHash} val input = psbt.getInput(pos) @@ -152,14 +164,17 @@ class LocalOnchainKeyManager(override val wallet: String, seed: ByteVector, time // update the input with the right script for a p2wpkh input, which is a * p2pkh * script // then sign and finalize the psbt input - val updated = psbt.updateWitnessInput(psbt.getGlobal.getTx.txIn.get(pos).outPoint, input.getWitnessUtxo, null, Script.pay2pkh(pub), SigHash.SIGHASH_ALL, input.getDerivationPaths) - val signed = EitherKt.flatMap(updated, (p: Psbt) => p.sign(priv, pos)) - val finalized = EitherKt.flatMap(signed, (s: SignPsbtResult) => { + + val updated: Either[UpdateFailure, Psbt] = psbt.updateWitnessInput(psbt.getGlobal.getTx.txIn.get(pos).outPoint, input.getWitnessUtxo, null, Script.pay2pkh(pub), SigHash.SIGHASH_ALL, input.getDerivationPaths) + val signed = updated.flatMap(_.sign(priv, pos)) + val finalized = signed.flatMap(s => { val sig = s.getSig require(sig.get(sig.size() - 1).toInt == SigHash.SIGHASH_ALL, "signature must end with SIGHASH_ALL") s.getPsbt.finalizeWitnessInput(pos, new ScriptWitness().push(sig).push(pub.value)) }) - require(finalized.isRight, s"cannot sign psbt input, error = ${finalized.getLeft}") - finalized.getRight + finalized match { + case Right(psbt) => psbt + case Left(failure) => throw new RuntimeException(s"cannot sign psbt input, error = $failure") + } } } diff --git a/eclair-core/src/main/scala/fr/acinq/eclair/crypto/keymanager/OnchainKeyManager.scala b/eclair-core/src/main/scala/fr/acinq/eclair/crypto/keymanager/OnchainKeyManager.scala index 0cc7b876aa..07008e70bf 100644 --- a/eclair-core/src/main/scala/fr/acinq/eclair/crypto/keymanager/OnchainKeyManager.scala +++ b/eclair-core/src/main/scala/fr/acinq/eclair/crypto/keymanager/OnchainKeyManager.scala @@ -3,8 +3,11 @@ package fr.acinq.eclair.crypto.keymanager import fr.acinq.bitcoin.psbt.Psbt import fr.acinq.bitcoin.scalacompat.Crypto.PublicKey import fr.acinq.bitcoin.scalacompat.DeterministicWallet.KeyPath +import fr.acinq.eclair.TimestampSecond import fr.acinq.eclair.blockchain.bitcoind.rpc.BitcoinCoreClient.Descriptors +import scala.util.Try + trait OnchainKeyManager { def wallet: String @@ -22,6 +25,12 @@ trait OnchainKeyManager { */ def getPublicKey(keyPath: KeyPath): (PublicKey, String) + /** + * + * @return the creation time of the wallet managed by this key manager + */ + def getWalletTimestamp(): TimestampSecond + /** * * @param account account number @@ -36,5 +45,5 @@ trait OnchainKeyManager { * @param ourOutputs index of outputs that belong to our onchain wallet * @return a signed psbt, where all our inputs are signed */ - def signPsbt(psbt: Psbt, ourInputs: Seq[Int], ourOutputs: Seq[Int]): Psbt + def signPsbt(psbt: Psbt, ourInputs: Seq[Int], ourOutputs: Seq[Int]): Try[Psbt] } diff --git a/eclair-core/src/test/scala/fr/acinq/eclair/blockchain/bitcoind/BitcoinCoreClientSpec.scala b/eclair-core/src/test/scala/fr/acinq/eclair/blockchain/bitcoind/BitcoinCoreClientSpec.scala index 0c3283f7dc..073cb06fdb 100644 --- a/eclair-core/src/test/scala/fr/acinq/eclair/blockchain/bitcoind/BitcoinCoreClientSpec.scala +++ b/eclair-core/src/test/scala/fr/acinq/eclair/blockchain/bitcoind/BitcoinCoreClientSpec.scala @@ -20,10 +20,9 @@ import akka.actor.Status.Failure import akka.pattern.pipe import akka.testkit.TestProbe import fr.acinq.bitcoin -import fr.acinq.bitcoin.psbt.Psbt +import fr.acinq.bitcoin.psbt.{Psbt, UpdateFailure} import fr.acinq.bitcoin.scalacompat.Crypto.PublicKey import fr.acinq.bitcoin.scalacompat.{Block, Btc, BtcDouble, ByteVector32, Crypto, DeterministicWallet, MilliBtcDouble, MnemonicCode, OP_DROP, OP_PUSHDATA, OutPoint, Satoshi, SatoshiLong, Script, ScriptWitness, Transaction, TxIn, TxOut, addressFromPublicKeyScript, addressToPublicKeyScript, computeBIP84Address, computeP2PkhAddress, computeP2WpkhAddress} -import fr.acinq.bitcoin.utils.EitherKt import fr.acinq.bitcoin.{Bech32, SigHash, SigVersion} import fr.acinq.eclair.blockchain.OnChainWallet.{FundTransactionResponse, MakeFundingTxResponse, OnChainBalance} import fr.acinq.eclair.blockchain.WatcherSpec.{createSpendManyP2WPKH, createSpendP2WPKH} @@ -202,7 +201,6 @@ class BitcoinCoreClientSpec extends TestKitBaseClass with BitcoindService with A test("fund transactions with external inputs") { import fr.acinq.bitcoin.scalacompat.KotlinUtils._ - import fr.acinq.bitcoin.utils.EitherKt val sender = TestProbe() val defaultWallet = makeBitcoinCoreClient @@ -263,10 +261,9 @@ class BitcoinCoreClientSpec extends TestKitBaseClass with BitcoindService with A // And let bitcoind sign the wallet input. walletExternalFunds.signPsbt(new Psbt(fundedTx2.tx), fundedTx2.tx.txIn.indices, Nil).pipeTo(sender.ref) val psbt = sender.expectMsgType[ProcessPsbtResponse].psbt - val updated = psbt.updateWitnessInput(outpoint1, txOut1, null, null, null, java.util.Map.of()) - val finalized = EitherKt.flatMap(updated, (psbt: Psbt) => psbt.finalizeWitnessInput(0, Script.witnessMultiSigMofN(Seq(alicePriv, bobPriv).map(_.publicKey), Seq(externalSig)))) - val psbt1: Psbt = finalized.getRight - val signedTx: Transaction = psbt1.extract().getRight + val updated: Either[UpdateFailure, Psbt] = psbt.updateWitnessInput(outpoint1, txOut1, null, null, null, java.util.Map.of()) + val finalized = updated.flatMap(_.finalizeWitnessInput(0, Script.witnessMultiSigMofN(Seq(alicePriv, bobPriv).map(_.publicKey), Seq(externalSig)))) + val signedTx: Transaction = finalized.flatMap(_.extract()).getOrElse(throw new RuntimeException("cannot compute transaction")) walletExternalFunds.publishTransaction(signedTx).pipeTo(sender.ref) sender.expectMsg(signedTx.txid) @@ -295,9 +292,10 @@ class BitcoinCoreClientSpec extends TestKitBaseClass with BitcoindService with A // We sign our external input. val externalSig = Transaction.signInput(fundedTx.tx, 0, inputScript2, SigHash.SIGHASH_ALL, 300_000 sat, SigVersion.SIGVERSION_WITNESS_V0, alicePriv) - val updated = psbt.updateWitnessInput(externalOutpoint, tx2.txOut(0), null, null, null, java.util.Map.of()) - val finalized = EitherKt.flatMap(updated, (psbt: Psbt) => psbt.finalizeWitnessInput(0, Script.witnessMultiSigMofN(Seq(alicePriv, carolPriv).map(_.publicKey), Seq(externalSig)))) - val signedTx: Transaction = finalized.getRight.extract().getRight + + val updated: Either[UpdateFailure, Psbt] = psbt.updateWitnessInput(externalOutpoint, tx2.txOut(0), null, null, null, java.util.Map.of()) + val finalized = updated.flatMap(_.finalizeWitnessInput(0, Script.witnessMultiSigMofN(Seq(alicePriv, carolPriv).map(_.publicKey), Seq(externalSig)))) + val signedTx: Transaction = finalized.flatMap(_.extract()).getOrElse(throw new RuntimeException("cannot compute transaction")) walletExternalFunds.publishTransaction(signedTx).pipeTo(sender.ref) sender.expectMsg(signedTx.txid) @@ -333,9 +331,9 @@ class BitcoinCoreClientSpec extends TestKitBaseClass with BitcoindService with A // We sign our external input. val externalSig = Transaction.signInput(fundedTx.tx, 0, inputScript2, SigHash.SIGHASH_ALL, 300_000 sat, SigVersion.SIGVERSION_WITNESS_V0, alicePriv) - val updated = psbt.updateWitnessInput(OutPoint(tx2, 0), tx2.txOut(0), null, null, null, java.util.Map.of()) - val finalized = EitherKt.flatMap(updated, (psbt: Psbt) => psbt.finalizeWitnessInput(0, Script.witnessMultiSigMofN(Seq(alicePriv, carolPriv).map(_.publicKey), Seq(externalSig)))) - val signedTx: Transaction = finalized.getRight.extract().getRight + val updated: Either[UpdateFailure, Psbt] = psbt.updateWitnessInput(OutPoint(tx2, 0), tx2.txOut(0), null, null, null, java.util.Map.of()) + val finalized = updated.flatMap(_.finalizeWitnessInput(0, Script.witnessMultiSigMofN(Seq(alicePriv, carolPriv).map(_.publicKey), Seq(externalSig)))) + val signedTx: Transaction = finalized.flatMap(_.extract()).getOrElse(throw new RuntimeException("cannot compute transaction")) walletExternalFunds.publishTransaction(signedTx).pipeTo(sender.ref) sender.expectMsg(signedTx.txid) // We have replaced the previous transaction. @@ -684,9 +682,9 @@ class BitcoinCoreClientSpec extends TestKitBaseClass with BitcoindService with A val nonWalletWitness = ScriptWitness(Seq(nonWalletSig, nonWalletKey.publicKey.value)) val txWithSignedNonWalletInput = txWithNonWalletInput.updateWitness(0, nonWalletWitness) val psbt = new Psbt(txWithSignedNonWalletInput) - val updated = psbt.updateWitnessInput(psbt.getGlobal.getTx.txIn.get(0).outPoint, txToRemote.txOut(0), null, fr.acinq.bitcoin.Script.pay2pkh(nonWalletKey.publicKey), SigHash.SIGHASH_ALL, psbt.getInput(0).getDerivationPaths) - val finalized = EitherKt.flatMap(updated, (psbt: Psbt) => psbt.finalizeWitnessInput(0, nonWalletWitness)) - bitcoinClient.signPsbt(finalized.getRight, txWithSignedNonWalletInput.txIn.indices.tail, Nil).pipeTo(sender.ref) + val updated: Either[UpdateFailure, Psbt] = psbt.updateWitnessInput(psbt.getGlobal.getTx.txIn.get(0).outPoint, txToRemote.txOut(0), null, fr.acinq.bitcoin.Script.pay2pkh(nonWalletKey.publicKey), SigHash.SIGHASH_ALL, psbt.getInput(0).getDerivationPaths) + val Right(psbt1) = updated.flatMap(_.finalizeWitnessInput(0, nonWalletWitness)) + bitcoinClient.signPsbt(psbt1, txWithSignedNonWalletInput.txIn.indices.tail, Nil).pipeTo(sender.ref) val signTxResponse2 = sender.expectMsgType[ProcessPsbtResponse] assert(signTxResponse2.complete) Transaction.correctlySpends(signTxResponse2.finalTx, txToRemote +: walletInputTxs, bitcoin.ScriptFlags.STANDARD_SCRIPT_VERIFY_FLAGS) @@ -722,9 +720,9 @@ class BitcoinCoreClientSpec extends TestKitBaseClass with BitcoindService with A val txWithSignedUnconfirmedInput = txWithUnconfirmedInput.updateWitness(0, nonWalletWitness) val previousTx = PreviousTx(Transactions.InputInfo(OutPoint(unconfirmedTx.txid, 0), unconfirmedTx.txOut.head, Script.pay2pkh(nonWalletKey.publicKey)), nonWalletWitness) val psbt = new Psbt(txWithSignedUnconfirmedInput) - val updated = psbt.updateWitnessInput(psbt.getGlobal.getTx.txIn.get(0).outPoint, unconfirmedTx.txOut(0), null, fr.acinq.bitcoin.Script.pay2pkh(nonWalletKey.publicKey), SigHash.SIGHASH_ALL, psbt.getInput(0).getDerivationPaths) - val finalized = EitherKt.flatMap(updated, (psbt: Psbt) => psbt.finalizeWitnessInput(0, nonWalletWitness)) - bitcoinClient.signPsbt(finalized.getRight, txWithSignedUnconfirmedInput.txIn.indices.tail, Nil).pipeTo(sender.ref) + val updated: Either[UpdateFailure, Psbt] = psbt.updateWitnessInput(psbt.getGlobal.getTx.txIn.get(0).outPoint, unconfirmedTx.txOut(0), null, fr.acinq.bitcoin.Script.pay2pkh(nonWalletKey.publicKey), SigHash.SIGHASH_ALL, psbt.getInput(0).getDerivationPaths) + val Right(psbt1) = updated.flatMap(_.finalizeWitnessInput(0, nonWalletWitness)) + bitcoinClient.signPsbt(psbt1, txWithSignedUnconfirmedInput.txIn.indices.tail, Nil).pipeTo(sender.ref) assert(sender.expectMsgType[ProcessPsbtResponse].complete) } } @@ -944,7 +942,7 @@ class BitcoinCoreClientSpec extends TestKitBaseClass with BitcoindService with A bitcoinClient.getMempoolTx(tx.txid).pipeTo(sender.ref) val mempoolTx = sender.expectMsgType[MempoolTx] - val currentFeerate = FeeratePerKw(mempoolTx.fees * 1000 / tx.weight()) + val currentFeerate = Transactions.fee2rate(mempoolTx.fees, tx.weight()) bitcoinClient.getMempoolPackage(Set(tx.txid)).pipeTo(sender.ref) sender.expectMsg(Map(tx.txid -> mempoolTx)) @@ -1128,7 +1126,7 @@ class BitcoinCoreClientSpec extends TestKitBaseClass with BitcoindService with A assert(tx.txOut.length == 2) // there must be a change output bitcoinClient.getMempoolTx(tx.txid).pipeTo(sender.ref) val mempoolTx = sender.expectMsgType[MempoolTx] - val currentFeerate = FeeratePerKw(mempoolTx.fees * 1000 / tx.weight()) + val currentFeerate = Transactions.fee2rate(mempoolTx.fees, tx.weight()) val targetFeerate = currentFeerate * 1.5 bitcoinClient.cpfp(Set(OutPoint(tx, 3)), targetFeerate).pipeTo(sender.ref) @@ -1442,7 +1440,7 @@ class BitcoinCoreClientWithEclairSignerSpec extends BitcoinCoreClientSpec { val onchainKeyManager = new LocalOnchainKeyManager(s"eclair_$hex", seed, TimestampSecond.now(), Block.RegtestGenesisBlock.hash) val jsonRpcClient = new BasicBitcoinJsonRPCClient(rpcAuthMethod = bitcoinrpcauthmethod, host = "localhost", port = bitcoindRpcPort, wallet = Some(s"eclair_$hex")) val wallet1 = new BitcoinCoreClient(jsonRpcClient, Some(onchainKeyManager)) - wallet1.createDescriptorWallet().pipeTo(sender.ref) + wallet1.createEclairBackedWallet().pipeTo(sender.ref) sender.expectMsg(true) // this account xpub can be used to create a watch-only wallet @@ -1482,7 +1480,7 @@ class BitcoinCoreClientWithEclairSignerSpec extends BitcoinCoreClientSpec { val onchainKeyManager = new LocalOnchainKeyManager(s"eclair_$hex", seed, TimestampSecond.now(), Block.RegtestGenesisBlock.hash) val jsonRpcClient = new BasicBitcoinJsonRPCClient(rpcAuthMethod = bitcoinrpcauthmethod, host = "localhost", port = bitcoindRpcPort, wallet = Some(s"eclair_$hex")) val wallet1 = new BitcoinCoreClient(jsonRpcClient, Some(onchainKeyManager)) - wallet1.createDescriptorWallet().pipeTo(sender.ref) + wallet1.createEclairBackedWallet().pipeTo(sender.ref) sender.expectMsg(true) wallet1.getReceiveAddress().pipeTo(sender.ref) val address = sender.expectMsgType[String] diff --git a/eclair-core/src/test/scala/fr/acinq/eclair/blockchain/bitcoind/BitcoindService.scala b/eclair-core/src/test/scala/fr/acinq/eclair/blockchain/bitcoind/BitcoindService.scala index 39011ab6ff..9d4c2426b7 100644 --- a/eclair-core/src/test/scala/fr/acinq/eclair/blockchain/bitcoind/BitcoindService.scala +++ b/eclair-core/src/test/scala/fr/acinq/eclair/blockchain/bitcoind/BitcoindService.scala @@ -174,7 +174,7 @@ trait BitcoindService extends Logging { val sender = TestProbe() waitForBitcoindUp(sender) if (useEclairSigner) { - makeBitcoinCoreClient.createDescriptorWallet().pipeTo(sender.ref) + makeBitcoinCoreClient.createEclairBackedWallet().pipeTo(sender.ref) sender.expectMsg(true) } else { sender.send(bitcoincli, BitcoinReq("createwallet", defaultWallet)) diff --git a/eclair-core/src/test/scala/fr/acinq/eclair/channel/publish/ReplaceableTxFunderSpec.scala b/eclair-core/src/test/scala/fr/acinq/eclair/channel/publish/ReplaceableTxFunderSpec.scala index 431aa70526..e65a69e154 100644 --- a/eclair-core/src/test/scala/fr/acinq/eclair/channel/publish/ReplaceableTxFunderSpec.scala +++ b/eclair-core/src/test/scala/fr/acinq/eclair/channel/publish/ReplaceableTxFunderSpec.scala @@ -70,13 +70,13 @@ class ReplaceableTxFunderSpec extends TestKitBaseClass with AnyFunSuiteLike { paymentHash, 17, ConfirmationTarget.Absolute(BlockHeight(0)) - ), PlaceHolderSig, preimage, Nil, Nil) + ), PlaceHolderSig, preimage) val htlcTimeout = HtlcTimeoutWithWitnessData(HtlcTimeoutTx( InputInfo(OutPoint(commitTx, 1), commitTx.txOut.last, htlcTimeoutScript), Transaction(2, Seq(TxIn(OutPoint(commitTx, 1), ByteVector.empty, 0)), Seq(TxOut(4000 sat, Script.pay2wpkh(PlaceHolderPubKey))), 0), 12, ConfirmationTarget.Absolute(BlockHeight(0)) - ), PlaceHolderSig, Nil, Nil) + ), PlaceHolderSig) (htlcSuccess, htlcTimeout) } @@ -129,7 +129,7 @@ class ReplaceableTxFunderSpec extends TestKitBaseClass with AnyFunSuiteLike { test("adjust previous anchor transaction outputs") { val (commitTx, initialAnchorTx) = createAnchorTx() - val previousAnchorTx = ClaimLocalAnchorWithWitnessData(initialAnchorTx, Nil, Nil).updateTx(initialAnchorTx.tx.copy( + val previousAnchorTx = ClaimLocalAnchorWithWitnessData(initialAnchorTx).updateTx(initialAnchorTx.tx.copy( txIn = Seq( initialAnchorTx.tx.txIn.head, // The previous funding attempt added two wallet inputs: diff --git a/eclair-core/src/test/scala/fr/acinq/eclair/channel/publish/ReplaceableTxPublisherSpec.scala b/eclair-core/src/test/scala/fr/acinq/eclair/channel/publish/ReplaceableTxPublisherSpec.scala index adbfbaadd1..507d3a248d 100644 --- a/eclair-core/src/test/scala/fr/acinq/eclair/channel/publish/ReplaceableTxPublisherSpec.scala +++ b/eclair-core/src/test/scala/fr/acinq/eclair/channel/publish/ReplaceableTxPublisherSpec.scala @@ -1665,7 +1665,7 @@ class ReplaceableTxPublisherWithEclairSignerSpec extends ReplaceableTxPublisherS override def getP2wpkhPubkey(renew: Boolean): PublicKey = pubkey } - walletClient.createDescriptorWallet().pipeTo(probe.ref) + walletClient.createEclairBackedWallet().pipeTo(probe.ref) probe.expectMsg(true) (walletRpcClient, walletClient) diff --git a/eclair-core/src/test/scala/fr/acinq/eclair/crypto/keymanager/LocalOnchainKeyManagerSpec.scala b/eclair-core/src/test/scala/fr/acinq/eclair/crypto/keymanager/LocalOnchainKeyManagerSpec.scala index a59282d7e5..0c5c7c26fb 100644 --- a/eclair-core/src/test/scala/fr/acinq/eclair/crypto/keymanager/LocalOnchainKeyManagerSpec.scala +++ b/eclair-core/src/test/scala/fr/acinq/eclair/crypto/keymanager/LocalOnchainKeyManagerSpec.scala @@ -9,6 +9,7 @@ import scodec.bits.ByteVector import java.util.Base64 import scala.jdk.CollectionConverters.SeqHasAsJava +import scala.util.{Failure, Success} class LocalOnchainKeyManagerSpec extends AnyFunSuite { test("sign psbt (non-reg test)") { @@ -20,14 +21,13 @@ class LocalOnchainKeyManagerSpec extends AnyFunSuite { Base64.getDecoder.decode("cHNidP8BAHECAAAAAfZo4nGIyTg77MFmEBkQH1Au3Jl8vzB2WWQGGz/MbyssAAAAAAD9////ArAHPgUAAAAAFgAU6j9yVvLg66Zu3GM/xHbmXT0yvyiAlpgAAAAAABYAFODscQh3N7lmDYyV5yrHpGL2Zd4JAAAAAAABAH0CAAAAAaNdmqUNlziIjSaif3JUcvJWdyF0U5bYq13NMe+LbaBZAAAAAAD9////AjSp1gUAAAAAFgAUjfFMfBg8ulo/874n3+0ode7ka0BAQg8AAAAAACIAIPUn/XU17DfnvDkj8gn2twG3jtr2Z7sthy9K2MPTdYkaAAAAAAEBHzSp1gUAAAAAFgAUjfFMfBg8ulo/874n3+0ode7ka0AiBgM+PDdyxsVisa66SyBxiUvhEam8lEP64yujvVsEcGaqIxgPCfOBVAAAgAEAAIAAAACAAQAAAAMAAAAAIgIDWmAhb/sCV9+HjwFpPuy2TyEBi/Y11wrEHZUihe3N80EYDwnzgVQAAIABAACAAAAAgAEAAAAFAAAAAAA=") ).getRight - val psbt1 = onchainKeyManager.signPsbt(psbt, psbt.getInputs.toArray().indices, Seq(0)) + val Success(psbt1) = onchainKeyManager.signPsbt(psbt, psbt.getInputs.toArray().indices, Seq(0)) val tx = psbt1.extract() assert(tx.isRight) } test("sign psbt") { import fr.acinq.bitcoin.scalacompat.KotlinUtils._ - import fr.acinq.bitcoin.utils.EitherKt val seed = ByteVector.fromValidHex("01" * 32) val onchainKeyManager = new LocalOnchainKeyManager("eclair", seed, TimestampSecond.now(), Block.TestnetGenesisBlock.hash) @@ -53,26 +53,26 @@ class LocalOnchainKeyManagerSpec extends AnyFunSuite { val tx = Transaction(version = 2, txIn = utxos.map(tx => TxIn(OutPoint(tx, 0), Nil, fr.acinq.bitcoin.TxIn.SEQUENCE_FINAL)), txOut = TxOut(Satoshi(1000_000), Script.pay2wpkh(getPublicKey(0))) :: Nil, lockTime = 0) - val psbt: Psbt = { - val p0 = new Psbt(tx).updateWitnessInput(OutPoint(utxos(0), 0), utxos(0).txOut(0), null, Script.pay2pkh(getPublicKey(0)).map(scala2kmp).asJava, null, java.util.Map.of(getPublicKey(0), bip32paths(0))) - val p1 = EitherKt.flatMap(p0, (psbt: Psbt) => psbt.updateWitnessInput(OutPoint(utxos(1), 0), utxos(1).txOut(0), null, Script.pay2pkh(getPublicKey(1)).map(scala2kmp).asJava, null, java.util.Map.of(getPublicKey(1), bip32paths(1)))) - val p2 = EitherKt.flatMap(p1, (psbt: Psbt) => psbt.updateWitnessInput(OutPoint(utxos(2), 0), utxos(2).txOut(0), null, Script.pay2pkh(getPublicKey(2)).map(scala2kmp).asJava, null, java.util.Map.of(getPublicKey(2), bip32paths(2)))) - val p3 = EitherKt.flatMap(p2, (psbt: Psbt) => psbt.updateNonWitnessInput(utxos(0), 0, null, null, java.util.Map.of())) - val p4 = EitherKt.flatMap(p3, (psbt: Psbt) => psbt.updateNonWitnessInput(utxos(1), 0, null, null, java.util.Map.of())) - val p5 = EitherKt.flatMap(p4, (psbt: Psbt) => psbt.updateNonWitnessInput(utxos(2), 0, null, null, java.util.Map.of())) - val p6 = EitherKt.flatMap(p5, (psbt: Psbt) => psbt.updateWitnessOutput(0, null, null, java.util.Map.of(getPublicKey(0), bip32paths(0)))) - p6.getRight - } + + val Right(psbt) = for { + p0 <- new Psbt(tx).updateWitnessInput(OutPoint(utxos(0), 0), utxos(0).txOut(0), null, Script.pay2pkh(getPublicKey(0)).map(scala2kmp).asJava, null, java.util.Map.of(getPublicKey(0), bip32paths(0))) + p1 <- p0.updateWitnessInput(OutPoint(utxos(1), 0), utxos(1).txOut(0), null, Script.pay2pkh(getPublicKey(1)).map(scala2kmp).asJava, null, java.util.Map.of(getPublicKey(1), bip32paths(1))) + p2 <- p1.updateWitnessInput(OutPoint(utxos(2), 0), utxos(2).txOut(0), null, Script.pay2pkh(getPublicKey(2)).map(scala2kmp).asJava, null, java.util.Map.of(getPublicKey(2), bip32paths(2))) + p3 <- p2.updateNonWitnessInput(utxos(0), 0, null, null, java.util.Map.of()) + p4 <- p3.updateNonWitnessInput(utxos(1), 0, null, null, java.util.Map.of()) + p5 <- p4.updateNonWitnessInput(utxos(2), 0, null, null, java.util.Map.of()) + p6 <- p5.updateWitnessOutput(0, null, null, java.util.Map.of(getPublicKey(0), bip32paths(0))) + } yield p6 { // sign all inputs and outputs - val psbt1 = onchainKeyManager.signPsbt(psbt, Seq(0, 1, 2), Seq(0)) + val Success(psbt1) = onchainKeyManager.signPsbt(psbt, Seq(0, 1, 2), Seq(0)) val signedTx = psbt1.extract().getRight Transaction.correctlySpends(signedTx, utxos, ScriptFlags.STANDARD_SCRIPT_VERIFY_FLAGS) } { // sign the first 2 inputs only - val psbt1 = onchainKeyManager.signPsbt(psbt, Seq(0, 1), Seq(0)) + val Success(psbt1) = onchainKeyManager.signPsbt(psbt, Seq(0, 1), Seq(0)) // extracting the final tx fails because no all inputs as signed assert(psbt1.extract().isLeft) assert(psbt1.getInput(2).getScriptWitness == null) @@ -80,49 +80,39 @@ class LocalOnchainKeyManagerSpec extends AnyFunSuite { { // provide a wrong derivation path for the first input val updated = psbt.updateWitnessInput(OutPoint(utxos(0), 0), utxos(0).txOut(0), null, Script.pay2pkh(getPublicKey(0)).map(scala2kmp).asJava, null, java.util.Map.of(getPublicKey(0), bip32paths(2))).getRight // wrong bip32 path - val error = intercept[IllegalArgumentException] { - onchainKeyManager.signPsbt(updated, Seq(0, 1, 2), Seq(0)) - } + val Failure(error) = onchainKeyManager.signPsbt(updated, Seq(0, 1, 2), Seq(0)) assert(error.getMessage.contains("cannot compute private key")) } { // provide a wrong derivation path for the first output val updated = psbt.updateWitnessOutput(0, null, null, java.util.Map.of(getPublicKey(0), bip32paths(1))).getRight // wrong path - val error = intercept[IllegalArgumentException] { - onchainKeyManager.signPsbt(updated, Seq(0, 1, 2), Seq(0)) - } + val Failure(error) = onchainKeyManager.signPsbt(updated, Seq(0, 1, 2), Seq(0)) assert(error.getMessage.contains("cannot compute public key")) } { // lie about the amount being spent val updated = psbt.updateWitnessInput(OutPoint(utxos(0), 0), utxos(0).txOut(0).copy(amount = Satoshi(10)), null, Script.pay2pkh(getPublicKey(0)).map(scala2kmp).asJava, null, java.util.Map.of(getPublicKey(0), bip32paths(0))).getRight - val error = intercept[IllegalArgumentException] { - onchainKeyManager.signPsbt(updated, Seq(0, 1, 2), Seq(0)) - } + val Failure(error) = onchainKeyManager.signPsbt(updated, Seq(0, 1, 2), Seq(0)) assert(error.getMessage.contains("utxo mismatch")) } { // do not provide non-witness utxo for utxo #2 - val psbt: Psbt = { - val p0 = new Psbt(tx).updateWitnessInput(OutPoint(utxos(0), 0), utxos(0).txOut(0), null, Script.pay2pkh(getPublicKey(0)).map(scala2kmp).asJava, null, java.util.Map.of(getPublicKey(0), bip32paths(0))) - val p1 = EitherKt.flatMap(p0, (psbt: Psbt) => psbt.updateWitnessInput(OutPoint(utxos(1), 0), utxos(1).txOut(0), null, Script.pay2pkh(getPublicKey(1)).map(scala2kmp).asJava, null, java.util.Map.of(getPublicKey(1), bip32paths(1)))) - val p2 = EitherKt.flatMap(p1, (psbt: Psbt) => psbt.updateWitnessInput(OutPoint(utxos(2), 0), utxos(2).txOut(0), null, Script.pay2pkh(getPublicKey(2)).map(scala2kmp).asJava, null, java.util.Map.of(getPublicKey(2), bip32paths(2)))) - val p3 = EitherKt.flatMap(p2, (psbt: Psbt) => psbt.updateNonWitnessInput(utxos(0), 0, null, null, java.util.Map.of())) - val p4 = EitherKt.flatMap(p3, (psbt: Psbt) => psbt.updateNonWitnessInput(utxos(1), 0, null, null, java.util.Map.of())) - val p5 = EitherKt.flatMap(p4, (psbt: Psbt) => psbt.updateWitnessOutput(0, null, null, java.util.Map.of(getPublicKey(0), bip32paths(0)))) - p5.getRight - } - val error = intercept[IllegalArgumentException] { - onchainKeyManager.signPsbt(psbt, Seq(0, 1, 2), Seq(0)) - } + val Right(psbt) = for { + p0 <- new Psbt(tx).updateWitnessInput(OutPoint(utxos(0), 0), utxos(0).txOut(0), null, Script.pay2pkh(getPublicKey(0)).map(scala2kmp).asJava, null, java.util.Map.of(getPublicKey(0), bip32paths(0))) + p1 <- p0.updateWitnessInput(OutPoint(utxos(1), 0), utxos(1).txOut(0), null, Script.pay2pkh(getPublicKey(1)).map(scala2kmp).asJava, null, java.util.Map.of(getPublicKey(1), bip32paths(1))) + p2 <- p1.updateWitnessInput(OutPoint(utxos(2), 0), utxos(2).txOut(0), null, Script.pay2pkh(getPublicKey(2)).map(scala2kmp).asJava, null, java.util.Map.of(getPublicKey(2), bip32paths(2))) + p3 <- p2.updateNonWitnessInput(utxos(0), 0, null, null, java.util.Map.of()) + p4 <- p3.updateNonWitnessInput(utxos(1), 0, null, null, java.util.Map.of()) + p5 <- p4.updateWitnessOutput(0, null, null, java.util.Map.of(getPublicKey(0), bip32paths(0))) + } yield p5 + + val Failure(error) = onchainKeyManager.signPsbt(psbt, Seq(0, 1, 2), Seq(0)) assert(error.getMessage.contains("non-witness utxo is missing")) } { // use sighash type != SIGHASH_ALL val updated = psbt.updateWitnessInput(OutPoint(utxos(0), 0), utxos(0).txOut(0), null, Script.pay2pkh(getPublicKey(0)).map(scala2kmp).asJava, SigHash.SIGHASH_SINGLE, java.util.Map.of(getPublicKey(0), bip32paths(0))).getRight - val error = intercept[IllegalArgumentException] { - onchainKeyManager.signPsbt(updated, Seq(0, 1, 2), Seq(0)) - } + val Failure(error) = onchainKeyManager.signPsbt(updated, Seq(0, 1, 2), Seq(0)) assert(error.getMessage.contains("input sighashtype must be SIGHASH_ALL")) } } diff --git a/eclair-core/src/test/scala/fr/acinq/eclair/integration/ChannelIntegrationSpec.scala b/eclair-core/src/test/scala/fr/acinq/eclair/integration/ChannelIntegrationSpec.scala index 29adfe7718..a3a6c14799 100644 --- a/eclair-core/src/test/scala/fr/acinq/eclair/integration/ChannelIntegrationSpec.scala +++ b/eclair-core/src/test/scala/fr/acinq/eclair/integration/ChannelIntegrationSpec.scala @@ -37,7 +37,7 @@ import fr.acinq.eclair.router.Router import fr.acinq.eclair.transactions.Transactions.{AnchorOutputsCommitmentFormat, TxOwner} import fr.acinq.eclair.transactions.{OutgoingHtlc, Scripts, Transactions} import fr.acinq.eclair.wire.protocol._ -import fr.acinq.eclair.{MilliSatoshi, MilliSatoshiLong, TestUtils, randomBytes32} +import fr.acinq.eclair.{MilliSatoshi, MilliSatoshiLong, randomBytes32} import org.json4s.JsonAST.{JString, JValue} import java.util.UUID @@ -455,12 +455,9 @@ abstract class ChannelIntegrationSpec extends IntegrationSpec { class StandardChannelIntegrationSpec extends ChannelIntegrationSpec { test("start eclair nodes") { - val mapA = Map("eclair.node-alias" -> "A", "eclair.channel.expiry-delta-blocks" -> 40, "eclair.channel.fulfill-safety-before-timeout-blocks" -> 12, "eclair.server.port" -> TestUtils.availablePort, "eclair.api.port" -> TestUtils.availablePort) - val mapB = Map("eclair.node-alias" -> "C", "eclair.channel.expiry-delta-blocks" -> 40, "eclair.channel.fulfill-safety-before-timeout-blocks" -> 12, "eclair.server.port" -> TestUtils.availablePort, "eclair.api.port" -> TestUtils.availablePort) - val mapC = Map("eclair.node-alias" -> "F", "eclair.channel.expiry-delta-blocks" -> 40, "eclair.channel.fulfill-safety-before-timeout-blocks" -> 12, "eclair.server.port" -> TestUtils.availablePort, "eclair.api.port" -> TestUtils.availablePort) - instantiateEclairNode("A", ConfigFactory.parseMap(mapA.asJava).withFallback(withDefaultCommitment).withFallback(commonConfig)) - instantiateEclairNode("C", ConfigFactory.parseMap(mapB.asJava).withFallback(withAnchorOutputs).withFallback(commonConfig)) - instantiateEclairNode("F", ConfigFactory.parseMap(mapC.asJava).withFallback(withDefaultCommitment).withFallback(commonConfig)) + instantiateEclairNode("A", ConfigFactory.parseMap(Map("eclair.node-alias" -> "A", "eclair.channel.expiry-delta-blocks" -> 40, "eclair.channel.fulfill-safety-before-timeout-blocks" -> 12, "eclair.server.port" -> (if (useEclairSigner) 29840 else 29740), "eclair.api.port" -> (if (useEclairSigner) 28190 else 28090)).asJava).withFallback(withDefaultCommitment).withFallback(commonConfig)) + instantiateEclairNode("C", ConfigFactory.parseMap(Map("eclair.node-alias" -> "C", "eclair.channel.expiry-delta-blocks" -> 40, "eclair.channel.fulfill-safety-before-timeout-blocks" -> 12, "eclair.server.port" -> (if (useEclairSigner) 29841 else 29741), "eclair.api.port" -> (if (useEclairSigner) 28191 else 28091)).asJava).withFallback(withAnchorOutputs).withFallback(commonConfig)) + instantiateEclairNode("F", ConfigFactory.parseMap(Map("eclair.node-alias" -> "F", "eclair.channel.expiry-delta-blocks" -> 40, "eclair.channel.fulfill-safety-before-timeout-blocks" -> 12, "eclair.server.port" -> (if (useEclairSigner) 29842 else 29742), "eclair.api.port" -> (if (useEclairSigner) 28192 else 28092)).asJava).withFallback(withDefaultCommitment).withFallback(commonConfig)) } test("connect nodes") { diff --git a/eclair-core/src/test/scala/fr/acinq/eclair/integration/IntegrationSpec.scala b/eclair-core/src/test/scala/fr/acinq/eclair/integration/IntegrationSpec.scala index a4f6a3bf1a..4da4266120 100644 --- a/eclair-core/src/test/scala/fr/acinq/eclair/integration/IntegrationSpec.scala +++ b/eclair-core/src/test/scala/fr/acinq/eclair/integration/IntegrationSpec.scala @@ -151,8 +151,7 @@ abstract class IntegrationSpec extends TestKitBaseClass with BitcoindService wit Files.writeString(datadir.toPath.resolve("eclair-signer.conf"), eclairSignerConf) } implicit val system: ActorSystem = ActorSystem(s"system-$name", config) - - val setup = new Setup(datadir, pluginParams = Seq.empty, seeds_opt = Some(Setup.Seeds(randomBytes32(), randomBytes32()))) + val setup = new Setup(datadir, pluginParams = Seq.empty) val kit = Await.result(setup.bootstrap, 10 seconds) nodes = nodes + (name -> kit) } diff --git a/eclair-core/src/test/scala/fr/acinq/eclair/integration/PaymentIntegrationSpec.scala b/eclair-core/src/test/scala/fr/acinq/eclair/integration/PaymentIntegrationSpec.scala index cab4e4735e..ea0604ab72 100644 --- a/eclair-core/src/test/scala/fr/acinq/eclair/integration/PaymentIntegrationSpec.scala +++ b/eclair-core/src/test/scala/fr/acinq/eclair/integration/PaymentIntegrationSpec.scala @@ -840,7 +840,7 @@ class PaymentIntegrationSpec extends IntegrationSpec { if (i % 10 == 0) { generateBlocks(1, Some(address)) } - AnnouncementsBatchValidationSpec.simulateChannel(bitcoinClient, onchainKeyManager) + AnnouncementsBatchValidationSpec.simulateChannel(bitcoinClient) } generateBlocks(1, Some(address)) logger.info(s"simulated ${channels.size} channels") diff --git a/eclair-core/src/test/scala/fr/acinq/eclair/router/AnnouncementsBatchValidationSpec.scala b/eclair-core/src/test/scala/fr/acinq/eclair/router/AnnouncementsBatchValidationSpec.scala index 093a8397ba..b5d7775a4f 100644 --- a/eclair-core/src/test/scala/fr/acinq/eclair/router/AnnouncementsBatchValidationSpec.scala +++ b/eclair-core/src/test/scala/fr/acinq/eclair/router/AnnouncementsBatchValidationSpec.scala @@ -19,19 +19,18 @@ package fr.acinq.eclair.router import akka.actor.ActorSystem import akka.pattern.pipe import akka.testkit.TestProbe +import sttp.client3.okhttp.OkHttpFutureBackend import fr.acinq.bitcoin.scalacompat.Crypto.PrivateKey import fr.acinq.bitcoin.scalacompat.{Block, Satoshi, SatoshiLong, Script, Transaction} import fr.acinq.eclair.blockchain.bitcoind.ZmqWatcher.ValidateResult import fr.acinq.eclair.blockchain.bitcoind.rpc.BitcoinJsonRPCAuthMethod.UserPassword import fr.acinq.eclair.blockchain.bitcoind.rpc.{BasicBitcoinJsonRPCClient, BitcoinCoreClient} import fr.acinq.eclair.blockchain.fee.FeeratePerKw -import fr.acinq.eclair.crypto.keymanager.OnchainKeyManager import fr.acinq.eclair.transactions.Scripts import fr.acinq.eclair.wire.protocol.{ChannelAnnouncement, ChannelUpdate} import fr.acinq.eclair.{BlockHeight, CltvExpiryDelta, Features, MilliSatoshiLong, RealShortChannelId, ShortChannelId, randomKey} import org.json4s.JsonAST.JString import org.scalatest.funsuite.AnyFunSuite -import sttp.client3.okhttp.OkHttpFutureBackend import scala.concurrent.duration._ import scala.concurrent.{Await, ExecutionContext} @@ -55,7 +54,7 @@ class AnnouncementsBatchValidationSpec extends AnyFunSuite { val channels = for (i <- 0 until 50) yield { // let's generate a block every 10 txs so that we can compute short ids if (i % 10 == 0) generateBlocks(1, bitcoinClient) - simulateChannel(bitcoinClient, null) + simulateChannel(bitcoinClient) } generateBlocks(6, bitcoinClient) val announcements = channels.map(c => makeChannelAnnouncement(c, bitcoinClient)) @@ -86,7 +85,7 @@ object AnnouncementsBatchValidationSpec { Await.result(generatedF, 10 seconds) } - def simulateChannel(bitcoinClient: BitcoinCoreClient, onchainKeyManager: OnchainKeyManager)(implicit ec: ExecutionContext): SimulatedChannel = { + def simulateChannel(bitcoinClient: BitcoinCoreClient)(implicit ec: ExecutionContext): SimulatedChannel = { val node1Key = randomKey() val node2Key = randomKey() val node1BitcoinKey = randomKey() From eff698a0c0343b9a85c2901b707b518488df8ccd Mon Sep 17 00:00:00 2001 From: Bastien Teinturier <31281497+t-bast@users.noreply.github.com> Date: Tue, 29 Aug 2023 15:59:38 +0200 Subject: [PATCH 4/8] Additional changes to delegate bitcoin core keys to eclair (#2726) Refactor the `BitcoinCoreClient` and `LocalOnChainKeyManager` to: - rely less on exceptions - use more idiomatic scala (reduce dependency on kotlin types) - provide more detailed logs We also simplify the `useEclairSigner` field in `BitcoinCoreClient`. The complexity of handling the case where there was an on-chain key manager but for a different wallet than the one configured isn't something that should be used, so it wasn't worth supporting. Some checks were inconsistent and are now unified: - checking the exact `scriptPubKey` in our outputs in TODO and TODO - we allow using `fundTransaction` with a tx that already includes a change output (which may happen when RBF-ing a transaction) - `getP2wpkhPubkeyHashForChange` didn't verify the returned key We completely separate the two cases in `signPsbt`, because otherwise in the non eclair-backed case, we were calling bitcoind's `processpsbt` twice for no good reason, which is bad for performance. We also decouple the `OnChainKeyManager` from the `BitcoinCoreClient`. This lets users keep running their eclair node with a bitcoin client that owns the private key while configuring the on-chain key manager for a future bitcoin client that will leverage this on-chain key manager. Users can use the eclair APIs to get the master xpub and descriptors to properly configure their next bitcoin core node, and switch to it once it has synchronized the descriptors. --- docs/BitcoinCoreKeys.md | 62 ++--- .../main/scala/fr/acinq/eclair/Eclair.scala | 34 +-- .../scala/fr/acinq/eclair/NodeParams.scala | 11 +- .../main/scala/fr/acinq/eclair/Setup.scala | 35 +-- .../eclair/blockchain/OnChainWallet.scala | 36 ++- .../bitcoind/rpc/BitcoinCoreClient.scala | 255 +++++++----------- .../channel/fund/InteractiveTxBuilder.scala | 32 +-- .../channel/publish/ReplaceableTxFunder.scala | 103 ++++--- .../keymanager/LocalOnChainKeyManager.scala | 239 ++++++++++++++++ .../keymanager/LocalOnchainKeyManager.scala | 180 ------------- .../crypto/keymanager/OnChainKeyManager.scala | 70 +++++ .../crypto/keymanager/OnchainKeyManager.scala | 49 ---- .../scala/fr/acinq/eclair/StartupSpec.scala | 2 +- .../scala/fr/acinq/eclair/TestConstants.scala | 2 + .../blockchain/DummyOnChainWallet.scala | 15 +- .../bitcoind/BitcoinCoreClientSpec.scala | 245 ++++++++--------- .../blockchain/bitcoind/BitcoindService.scala | 26 +- .../channel/InteractiveTxBuilderSpec.scala | 14 +- .../publish/ReplaceableTxPublisherSpec.scala | 8 +- ...scala => LocalOnChainKeyManagerSpec.scala} | 40 ++- .../basic/fixtures/MinimalNodeFixture.scala | 21 +- .../acinq/eclair/api/handlers/OnChain.scala | 7 +- 22 files changed, 738 insertions(+), 748 deletions(-) create mode 100644 eclair-core/src/main/scala/fr/acinq/eclair/crypto/keymanager/LocalOnChainKeyManager.scala delete mode 100644 eclair-core/src/main/scala/fr/acinq/eclair/crypto/keymanager/LocalOnchainKeyManager.scala create mode 100644 eclair-core/src/main/scala/fr/acinq/eclair/crypto/keymanager/OnChainKeyManager.scala delete mode 100644 eclair-core/src/main/scala/fr/acinq/eclair/crypto/keymanager/OnchainKeyManager.scala rename eclair-core/src/test/scala/fr/acinq/eclair/crypto/keymanager/{LocalOnchainKeyManagerSpec.scala => LocalOnChainKeyManagerSpec.scala} (83%) diff --git a/docs/BitcoinCoreKeys.md b/docs/BitcoinCoreKeys.md index b01f5c5b64..ba8b9de14f 100644 --- a/docs/BitcoinCoreKeys.md +++ b/docs/BitcoinCoreKeys.md @@ -6,22 +6,22 @@ is less secure than for Eclair (because it is shared among several services for ## Configuring Eclair and Bitcoin Core to use a new Eclair-backed bitcoin wallet -Follow these steps to delegate onchain key management to eclair: +Follow these steps to delegate on-chain key management to eclair: -1) Generate a BIP39 mnemonic code and passphrase +### 1. Generate a BIP39 mnemonic code and passphrase You can use any BIP39-compatible tool, including most hardware wallets. -2) Create an `eclair-signer.conf` configuration file add it to eclair's data directory +### 2. Create an `eclair-signer.conf` configuration file add it to eclair's data directory A signer configuration file uses the HOCON format that we already use for `eclair.conf` and must include the following options: - key | description + key | description --------------------------|-------------------------------------------------------------------------- - eclair.signer.wallet | wallet name - eclair.signer.mnemonics | BIP39 mnemonic words - eclair.signer.passphrase | passphrase - eclair.signer.timestamp | wallet creation UNIX timestamp. Set to the current time for new wallets. + eclair.signer.wallet | wallet name + eclair.signer.mnemonics | BIP39 mnemonic words + eclair.signer.passphrase | passphrase + eclair.signer.timestamp | wallet creation UNIX timestamp. Set to the current time for new wallets. This is an example of `eclair-signer.conf` configuration file: @@ -38,7 +38,7 @@ This is an example of `eclair-signer.conf` configuration file: } ``` -3) Configure Eclair to handle private keys for this wallet +### 3. Configure Eclair to handle private keys for this wallet Set `eclair.bitcoind.wallet` to the name of the wallet in your `eclair-signer.conf` file (`eclair` in the example above) and restart Eclair. Eclair will automatically create a new, empty, descriptor-enabled, watch-only wallet in Bitcoin Core and import its descriptors. @@ -48,45 +48,38 @@ passphrase that your are using are new, you can safely update this timestamp, bu the steps described in the next section. You now have a Bitcoin Core watch-only wallet for which only your Eclair node can sign transactions. This Bitcoin Core wallet can -safely be copied to another Bitcoin Core node to monitor your onchain funds. +safely be copied to another Bitcoin Core node to monitor your on-chain funds. You can also use `eclair-cli getmasterxpub` to get a BIP32 extended public key that you can import into any compatible Bitcoin wallet to create a watch-only wallet (Electrum for example) that you can use to monitor your Bitcoin Core balance. :warning: this means that your Bitcoin Core wallet cannot send funds on its own (since it cannot access private keys to sign transactions). -To send funds onchain you must use `eclair-cli sendonchain`. +To send funds on-chain you must use `eclair-cli sendonchain`. :warning: to backup the private keys of this wallet you must either backup your mnemonic code and passphrase, or backup the `eclair-signer.conf` file in your eclair directory (default is `~/.eclair`) along with your channels and node seed files. -:warning: You can also initialize a backup onchain wallet with the same mnemonic code and passphrase (on a hardware wallet for example), but be warned that using them may interfere with your node's operations (for example you may end up +:warning: You can also initialize a backup on-chain wallet with the same mnemonic code and passphrase (on a hardware wallet for example), but be warned that using them may interfere with your node's operations (for example you may end up double-spending funding transactions generated by your node). -## Importing an existing Eclair-backed bitcoin core wallet +## Importing an existing Eclair-backed Bitcoin Core wallet -The steps above describe how you can simply switch to a new Eclair-backed bitcoin wallet. -If you are already using an Eclair-backed bitcoin wallet that you want to move to another setup, you have several options: +The steps above described how you can simply switch to a new Eclair-backed bitcoin wallet. +Follow the steps below if you are already using an Eclair-backed bitcoin wallet that you want to move to another Bitcoin Core node. -### Copy the bitcoin core wallet and `eclair-signer.conf` +### 1. Create an empty, descriptor-enabled, watch-only wallet in Bitcoin Core -Copy your wallet file to a new Bitcoin Core node, and load it with `bitcoin-cli loadwallet`. Bitcoin Core may need to scan the blockchain which could take some time. -Copy `eclair-signer.conf` to your Eclair data directory and set `eclair.bitcoind.wallet` to the name of the wallet configured in `eclair-signer.conf` (and which should also be the name of your Bitcoin Core wallet). -Once your wallet has been imported, just restart Eclair. +Start by creating a watch-only wallet on your new Bitcoin Core node. -### Create an empty wallet manually and import Eclair descriptors - -1) Create an empty, descriptor-enabled, watch-only wallet in Bitcoin Core: - :warning: The name must match the one that you set in `eclair-signer.conf` (here we use "eclair") +:warning: The name must match the one that you set in `eclair-signer.conf` (here we use "eclair") ```shell $ bitcoin-cli -named createwallet wallet_name=eclair disable_private_keys=true blank=true descriptors=true load_on_startup=true ``` -2) Import public descriptors generated by Eclair - -Copy `eclair-signer.conf` to your Eclair data directory but do not change `eclair.bitcoind.wallet`, and restart Eclair. +### 2. Import public descriptors generated by Eclair -`eclair-cli getdescriptors` will return public wallet descriptors in a format that is compatible with Bitcoin Core, and that you can import with `bitcoin-cli -rpcwallet=eclair importdescriptors` +Calling `eclair-cli getdescriptors` on your existing Eclair node will return public wallet descriptors in a format that is compatible with Bitcoin Core. This is an example of descriptors generated by Eclair: ```json @@ -106,7 +99,7 @@ This is an example of descriptors generated by Eclair: ] ``` -You can generate the descriptors with your Eclair node and import them into a Bitcoin node with the following commands: +Generate the descriptors with your Eclair node and import them into a Bitcoin node with the following commands: ```shell $ eclair-cli getdescriptors | jq --raw-output -c > descriptors.json @@ -115,6 +108,15 @@ $ cat descriptors.json | xargs -0 bitcoin-cli -rpcwallet=eclair importdescriptor :warning: Importing descriptors can take a long time, and your Bitcoin Core node will not be usable until it's done -3) Configure Eclair to handle private keys for this wallet +### 3. Configure Eclair to use your new Bitcoin Core node + +Once your new Bitcoin Core node has finished importing the descriptors, it is ready to be used by Eclair. + +In your `eclair.conf`: + +- set `eclair.bitcoind.host` to the address of your new Bitcoin Core node +- set `eclair.bitcoind.wallet` to the name of the wallet in `eclair-signer.conf` +- set `eclair.bitcoind.zmqblock` and `eclair.bitcoind.zmqtx` to use your new Bitcoin Core node +- update other field in the `eclair.bitcoind` section if necessary (`rpcport`, `auth`, `rpcuser`, `rpcuser`, etc) -Set `eclair.bitcoind.wallet` to the name of the wallet in `eclair-signer.conf` and restart Eclair. +Restart Eclair and it will start using your new Bitcoin Core node. diff --git a/eclair-core/src/main/scala/fr/acinq/eclair/Eclair.scala b/eclair-core/src/main/scala/fr/acinq/eclair/Eclair.scala index f49d7df1a8..b7ee3bdee2 100644 --- a/eclair-core/src/main/scala/fr/acinq/eclair/Eclair.scala +++ b/eclair-core/src/main/scala/fr/acinq/eclair/Eclair.scala @@ -181,7 +181,7 @@ trait Eclair { def payOfferBlocking(offer: Offer, amount: MilliSatoshi, quantity: Long, externalId_opt: Option[String] = None, maxAttempts_opt: Option[Int] = None, maxFeeFlat_opt: Option[Satoshi] = None, maxFeePct_opt: Option[Double] = None, pathFindingExperimentName_opt: Option[String] = None, connectDirectly: Boolean = false)(implicit timeout: Timeout): Future[PaymentEvent] - def getOnchainMasterPubKey(account: Long): String + def getOnChainMasterPubKey(account: Long): String def getDescriptors(account: Long): Descriptors @@ -681,16 +681,16 @@ class EclairImpl(appKit: Kit) extends Eclair with Logging { } } - def payOfferInternal(offer: Offer, - amount: MilliSatoshi, - quantity: Long, - externalId_opt: Option[String], - maxAttempts_opt: Option[Int], - maxFeeFlat_opt: Option[Satoshi], - maxFeePct_opt: Option[Double], - pathFindingExperimentName_opt: Option[String], - connectDirectly: Boolean, - blocking: Boolean)(implicit timeout: Timeout): Future[Any] = { + private def payOfferInternal(offer: Offer, + amount: MilliSatoshi, + quantity: Long, + externalId_opt: Option[String], + maxAttempts_opt: Option[Int], + maxFeeFlat_opt: Option[Satoshi], + maxFeePct_opt: Option[Double], + pathFindingExperimentName_opt: Option[String], + connectDirectly: Boolean, + blocking: Boolean)(implicit timeout: Timeout): Future[Any] = { if (externalId_opt.exists(_.length > externalIdMaxLength)) { return Future.failed(new IllegalArgumentException(s"externalId is too long: cannot exceed $externalIdMaxLength characters")) } @@ -733,14 +733,14 @@ class EclairImpl(appKit: Kit) extends Eclair with Logging { payOfferInternal(offer, amount, quantity, externalId_opt, maxAttempts_opt, maxFeeFlat_opt, maxFeePct_opt, pathFindingExperimentName_opt, connectDirectly, blocking = true).mapTo[PaymentEvent] } - override def getDescriptors(account: Long): Descriptors = appKit.wallet match { - case bitcoinCoreClient: BitcoinCoreClient if bitcoinCoreClient.onchainKeyManager_opt.isDefined => bitcoinCoreClient.onchainKeyManager_opt.get.getDescriptors(account) - case _ => throw new RuntimeException("onchain seed is not configured") + override def getDescriptors(account: Long): Descriptors = appKit.nodeParams.onChainKeyManager_opt match { + case Some(keyManager) => keyManager.descriptors(account) + case _ => throw new RuntimeException("on-chain seed is not configured") } - override def getOnchainMasterPubKey(account: Long): String = appKit.wallet match { - case bitcoinCoreClient: BitcoinCoreClient if bitcoinCoreClient.onchainKeyManager_opt.isDefined => bitcoinCoreClient.onchainKeyManager_opt.get.getOnchainMasterPubKey(account) - case _ => throw new RuntimeException("onchain seed is not configured") + override def getOnChainMasterPubKey(account: Long): String = appKit.nodeParams.onChainKeyManager_opt match { + case Some(keyManager) => keyManager.masterPubKey(account) + case _ => throw new RuntimeException("on-chain seed is not configured") } override def stop(): Future[Unit] = { diff --git a/eclair-core/src/main/scala/fr/acinq/eclair/NodeParams.scala b/eclair-core/src/main/scala/fr/acinq/eclair/NodeParams.scala index 9b7ce7535c..8bdeedd873 100644 --- a/eclair-core/src/main/scala/fr/acinq/eclair/NodeParams.scala +++ b/eclair-core/src/main/scala/fr/acinq/eclair/NodeParams.scala @@ -25,7 +25,7 @@ import fr.acinq.eclair.channel.ChannelFlags import fr.acinq.eclair.channel.fsm.Channel import fr.acinq.eclair.channel.fsm.Channel.{ChannelConf, UnhandledExceptionStrategy} import fr.acinq.eclair.crypto.Noise.KeyPair -import fr.acinq.eclair.crypto.keymanager.{ChannelKeyManager, NodeKeyManager} +import fr.acinq.eclair.crypto.keymanager.{ChannelKeyManager, NodeKeyManager, OnChainKeyManager} import fr.acinq.eclair.db._ import fr.acinq.eclair.io.MessageRelay.{RelayAll, RelayChannelsOnly, RelayPolicy} import fr.acinq.eclair.io.PeerConnection @@ -33,8 +33,8 @@ import fr.acinq.eclair.message.OnionMessages.OnionMessageConfig import fr.acinq.eclair.payment.relay.Relayer.{AsyncPaymentsParams, RelayFees, RelayParams} import fr.acinq.eclair.router.Announcements.AddressException import fr.acinq.eclair.router.Graph.{HeuristicsConstants, WeightRatios} +import fr.acinq.eclair.router.Router._ import fr.acinq.eclair.router.{Graph, PathFindingExperimentConf} -import fr.acinq.eclair.router.Router.{MessageRouteParams, MultiPartParams, PathFindingConf, RouterConf, SearchBoundaries} import fr.acinq.eclair.tor.Socks5ProxyParams import fr.acinq.eclair.wire.protocol._ import grizzled.slf4j.Logging @@ -54,6 +54,7 @@ import scala.jdk.CollectionConverters._ */ case class NodeParams(nodeKeyManager: NodeKeyManager, channelKeyManager: ChannelKeyManager, + onChainKeyManager_opt: Option[OnChainKeyManager], instanceId: UUID, // a unique instance ID regenerated after each restart private val blockHeight: AtomicLong, private val feerates: AtomicReference[FeeratesPerKw], @@ -101,7 +102,7 @@ case class NodeParams(nodeKeyManager: NodeKeyManager, def currentFeerates: FeeratesPerKw = feerates.get() /** Only to be used in tests. */ - def setFeerates(value: FeeratesPerKw) = feerates.set(value) + def setFeerates(value: FeeratesPerKw): Unit = feerates.set(value) /** Returns the features that should be used in our init message with the given peer. */ def initFeaturesFor(nodeId: PublicKey): Features[InitFeature] = overrideInitFeatures.getOrElse(nodeId, features).initFeatures() @@ -211,7 +212,8 @@ object NodeParams extends Logging { } } - def makeNodeParams(config: Config, instanceId: UUID, nodeKeyManager: NodeKeyManager, channelKeyManager: ChannelKeyManager, + def makeNodeParams(config: Config, instanceId: UUID, + nodeKeyManager: NodeKeyManager, channelKeyManager: ChannelKeyManager, onChainKeyManager_opt: Option[OnChainKeyManager], torAddress_opt: Option[NodeAddress], database: Databases, blockHeight: AtomicLong, feerates: AtomicReference[FeeratesPerKw], pluginParams: Seq[PluginParams] = Nil): NodeParams = { // check configuration for keys that have been renamed @@ -475,6 +477,7 @@ object NodeParams extends Logging { NodeParams( nodeKeyManager = nodeKeyManager, channelKeyManager = channelKeyManager, + onChainKeyManager_opt = onChainKeyManager_opt, instanceId = instanceId, blockHeight = blockHeight, feerates = feerates, diff --git a/eclair-core/src/main/scala/fr/acinq/eclair/Setup.scala b/eclair-core/src/main/scala/fr/acinq/eclair/Setup.scala index f284cfb061..f1653bf615 100644 --- a/eclair-core/src/main/scala/fr/acinq/eclair/Setup.scala +++ b/eclair-core/src/main/scala/fr/acinq/eclair/Setup.scala @@ -34,7 +34,7 @@ import fr.acinq.eclair.blockchain.fee._ import fr.acinq.eclair.channel.Register import fr.acinq.eclair.channel.fsm.Channel import fr.acinq.eclair.crypto.WeakEntropyPool -import fr.acinq.eclair.crypto.keymanager.{LocalChannelKeyManager, LocalNodeKeyManager, LocalOnchainKeyManager} +import fr.acinq.eclair.crypto.keymanager.{LocalChannelKeyManager, LocalNodeKeyManager, LocalOnChainKeyManager} import fr.acinq.eclair.db.Databases.FileBackup import fr.acinq.eclair.db.FileBackupHandler.FileBackupParams import fr.acinq.eclair.db.{Databases, DbEventHandler, FileBackupHandler} @@ -121,7 +121,7 @@ class Setup(val datadir: File, // early checks PortChecker.checkAvailable(serverBindingAddress) - val onchainKeyManager_opt = LocalOnchainKeyManager.load(datadir, NodeParams.hashFromChain(chain)) + val onChainKeyManager_opt = LocalOnChainKeyManager.load(datadir, NodeParams.hashFromChain(chain)) val (bitcoin, bitcoinChainHash) = { val wallet = { @@ -142,18 +142,6 @@ class Setup(val datadir: File, port = config.getInt("bitcoind.rpcport"), wallet = wallet) - def createEclairBackedWallet(wallets: List[String]): Future[Boolean] = { - if (wallet.exists(name => wallets.contains(name))) { - // wallet already exists - Future.successful(true) - } else { - new BitcoinCoreClient(bitcoinClient, onchainKeyManager_opt).createEclairBackedWallet().recover { case e => - logger.error(s"cannot create descriptor wallet", e) - throw BitcoinWalletNotCreatedException(wallet.getOrElse("")) - } - } - } - val future = for { json <- bitcoinClient.invoke("getblockchaininfo").recover { case e => throw BitcoinRPCConnectionException(e) } // Make sure wallet support is enabled in bitcoind. @@ -161,8 +149,11 @@ class Setup(val datadir: File, .collect { case JArray(values) => values.map(value => value.extract[String]) } - walletCreated <- createEclairBackedWallet(wallets) - _ = assert(walletCreated, "Cannot create eclair-backed wallet, check logs for details") + eclairBackedWalletOk <- onChainKeyManager_opt match { + case Some(keyManager) if !wallets.contains(keyManager.wallet) => keyManager.createWallet(bitcoinClient) + case _ => Future.successful(true) + } + _ = assert(eclairBackedWalletOk || onChainKeyManager_opt.map(_.wallet) != wallet, s"cannot create eclair-backed wallet=${onChainKeyManager_opt.map(_.wallet)}, check logs for details") progress = (json \ "verificationprogress").extract[Double] ibd = (json \ "initialblockdownload").extract[Boolean] blocks = (json \ "blocks").extract[Long] @@ -201,7 +192,7 @@ class Setup(val datadir: File, logger.info(s"connecting to database with instanceId=$instanceId") val databases = Databases.init(config.getConfig("db"), instanceId, chaindir, db) - val nodeParams = NodeParams.makeNodeParams(config, instanceId, nodeKeyManager, channelKeyManager, initTor(), databases, blockHeight, feeratesPerKw, pluginParams) + val nodeParams = NodeParams.makeNodeParams(config, instanceId, nodeKeyManager, channelKeyManager, onChainKeyManager_opt, initTor(), databases, blockHeight, feeratesPerKw, pluginParams) logger.info(s"nodeid=${nodeParams.nodeId} alias=${nodeParams.alias}") assert(bitcoinChainHash == nodeParams.chainHash, s"chainHash mismatch (conf=${nodeParams.chainHash} != bitcoind=$bitcoinChainHash)") @@ -236,9 +227,7 @@ class Setup(val datadir: File, minFeeratePerByte = FeeratePerByte(Satoshi(config.getLong("on-chain-fees.min-feerate"))) smoothFeerateWindow = config.getInt("on-chain-fees.smoothing-window") feeProvider = nodeParams.chainHash match { - case Block.RegtestGenesisBlock.hash => - FallbackFeeProvider(ConstantFeeProvider(defaultFeerates) :: Nil, minFeeratePerByte) - case Block.SignetGenesisBlock.hash => + case Block.RegtestGenesisBlock.hash | Block.SignetGenesisBlock.hash => FallbackFeeProvider(ConstantFeeProvider(defaultFeerates) :: Nil, minFeeratePerByte) case _ => FallbackFeeProvider(SmoothFeeProvider(BitcoinCoreFeeProvider(bitcoin, defaultFeerates), smoothFeerateWindow) :: Nil, minFeeratePerByte) @@ -263,7 +252,7 @@ class Setup(val datadir: File, finalPubkey = new AtomicReference[PublicKey](null) pubkeyRefreshDelay = FiniteDuration(config.getDuration("bitcoind.final-pubkey-refresh-delay").getSeconds, TimeUnit.SECONDS) - bitcoinClient = new BitcoinCoreClient(bitcoin, onchainKeyManager_opt) with OnchainPubkeyCache { + bitcoinClient = new BitcoinCoreClient(bitcoin, if (bitcoin.wallet == onChainKeyManager_opt.map(_.wallet)) onChainKeyManager_opt else None) with OnchainPubkeyCache { val refresher: typed.ActorRef[OnchainPubkeyRefresher.Command] = system.spawn(Behaviors.supervise(OnchainPubkeyRefresher(this, finalPubkey, pubkeyRefreshDelay)).onFailure(typed.SupervisorStrategy.restart), name = "onchain-address-manager") override def getP2wpkhPubkey(renew: Boolean): PublicKey = { @@ -325,7 +314,7 @@ class Setup(val datadir: File, routerTimeout = after(FiniteDuration(config.getDuration("router.init-timeout").getSeconds, TimeUnit.SECONDS), using = system.scheduler)(Future.failed(new RuntimeException("Router initialization timed out"))) _ <- Future.firstCompletedOf(routerInitialized.future :: routerTimeout :: Nil) - _ = bitcoinClient.getReceiveAddress().map(address => logger.info(s"initial wallet address=$address")) + _ = bitcoinClient.getReceiveAddress().map(address => logger.info(s"initial address=$address for bitcoin wallet=${bitcoinClient.rpcClient.wallet.getOrElse("")}")) channelsListener = system.spawn(ChannelsListener(channelsListenerReady), name = "channels-listener") _ <- channelsListenerReady.future @@ -479,8 +468,6 @@ case class BitcoinDefaultWalletException(loaded: List[String]) extends RuntimeEx case class BitcoinWalletNotLoadedException(wallet: String, loaded: List[String]) extends RuntimeException(s"configured wallet \"$wallet\" not in the set of loaded bitcoind wallets: ${loaded.map("\"" + _ + "\"").mkString("[", ",", "]")}") -case class BitcoinWalletNotCreatedException(wallet: String) extends RuntimeException(s"configured wallet \"$wallet\" does not exist and could not be created.") - case object EmptyAPIPasswordException extends RuntimeException("must set a password for the json-rpc api") case object IncompatibleDBException extends RuntimeException("database is not compatible with this version of eclair") diff --git a/eclair-core/src/main/scala/fr/acinq/eclair/blockchain/OnChainWallet.scala b/eclair-core/src/main/scala/fr/acinq/eclair/blockchain/OnChainWallet.scala index 47c5bb0c0d..0138f857af 100644 --- a/eclair-core/src/main/scala/fr/acinq/eclair/blockchain/OnChainWallet.scala +++ b/eclair-core/src/main/scala/fr/acinq/eclair/blockchain/OnChainWallet.scala @@ -19,7 +19,6 @@ package fr.acinq.eclair.blockchain import fr.acinq.bitcoin.psbt.Psbt import fr.acinq.bitcoin.scalacompat.Crypto.PublicKey import fr.acinq.bitcoin.scalacompat.{ByteVector32, OutPoint, Satoshi, Transaction} -import fr.acinq.eclair.blockchain.bitcoind.rpc.BitcoinCoreClient.ProcessPsbtResponse import fr.acinq.eclair.blockchain.fee.FeeratePerKw import scodec.bits.ByteVector @@ -34,12 +33,13 @@ trait OnChainChannelFunder { import OnChainWallet._ - /** Fund the provided transaction by adding inputs (and a change output if necessary). */ - def fundTransaction(tx: Transaction, feeRate: FeeratePerKw, replaceable: Boolean, externalInputsWeight: Map[OutPoint, Long] = Map.empty)(implicit ec: ExecutionContext): Future[FundTransactionResponse] - /** - * sign a PSBT. Result may be partially signed: only inputs known to our bitcoin core wallet will be signed + * Fund the provided transaction by adding inputs (and a change output if necessary). + * Callers must verify that the resulting transaction isn't sending funds to unexpected addresses (malicious bitcoin node). */ + def fundTransaction(tx: Transaction, feeRate: FeeratePerKw, replaceable: Boolean, externalInputsWeight: Map[OutPoint, Long] = Map.empty)(implicit ec: ExecutionContext): Future[FundTransactionResponse] + + /** Sign a PSBT. Result may be partially signed: only inputs known to our bitcoin wallet will be signed. */ def signPsbt(psbt: Psbt, ourInputs: Seq[Int], ourOutputs: Seq[Int])(implicit ec: ExecutionContext): Future[ProcessPsbtResponse] /** @@ -125,4 +125,30 @@ object OnChainWallet { final case class FundTransactionResponse(tx: Transaction, fee: Satoshi, changePosition: Option[Int]) { val amountIn: Satoshi = fee + tx.txOut.map(_.amount).sum } + + final case class ProcessPsbtResponse(psbt: Psbt, complete: Boolean) { + + import fr.acinq.bitcoin.psbt.UpdateFailure + import fr.acinq.bitcoin.scalacompat.KotlinUtils._ + + /** Transaction with all available witnesses. */ + val partiallySignedTx: Transaction = { + var tx = psbt.getGlobal.getTx + for (i <- 0 until psbt.getInputs.size()) { + Option(psbt.getInputs.get(i).getScriptWitness).foreach { witness => + tx = tx.updateWitness(i, witness) + } + } + tx + } + + /** Extract a fully signed transaction if the psbt is finalized. */ + val finalTx_opt: Either[UpdateFailure, Transaction] = { + val extracted: Either[UpdateFailure, fr.acinq.bitcoin.Transaction] = psbt.extract() + extracted match { + case Left(f) => Left(f) + case Right(tx) => Right(tx) + } + } + } } diff --git a/eclair-core/src/main/scala/fr/acinq/eclair/blockchain/bitcoind/rpc/BitcoinCoreClient.scala b/eclair-core/src/main/scala/fr/acinq/eclair/blockchain/bitcoind/rpc/BitcoinCoreClient.scala index 6e646d9a9e..7ae6e641c2 100644 --- a/eclair-core/src/main/scala/fr/acinq/eclair/blockchain/bitcoind/rpc/BitcoinCoreClient.scala +++ b/eclair-core/src/main/scala/fr/acinq/eclair/blockchain/bitcoind/rpc/BitcoinCoreClient.scala @@ -16,16 +16,16 @@ package fr.acinq.eclair.blockchain.bitcoind.rpc -import fr.acinq.bitcoin.psbt.{Psbt, UpdateFailure} +import fr.acinq.bitcoin.psbt.Psbt import fr.acinq.bitcoin.scalacompat.Crypto.PublicKey import fr.acinq.bitcoin.scalacompat._ import fr.acinq.bitcoin.{Bech32, Block, SigHash} import fr.acinq.eclair.ShortChannelId.coordinates import fr.acinq.eclair.blockchain.OnChainWallet -import fr.acinq.eclair.blockchain.OnChainWallet.{FundTransactionResponse, MakeFundingTxResponse, OnChainBalance} +import fr.acinq.eclair.blockchain.OnChainWallet.{FundTransactionResponse, MakeFundingTxResponse, OnChainBalance, ProcessPsbtResponse} import fr.acinq.eclair.blockchain.bitcoind.ZmqWatcher.{GetTxWithMetaResponse, UtxoStatus, ValidateResult} import fr.acinq.eclair.blockchain.fee.{FeeratePerKB, FeeratePerKw} -import fr.acinq.eclair.crypto.keymanager.OnchainKeyManager +import fr.acinq.eclair.crypto.keymanager.OnChainKeyManager import fr.acinq.eclair.transactions.Transactions import fr.acinq.eclair.wire.protocol.ChannelAnnouncement import fr.acinq.eclair.{BlockHeight, TimestampSecond, TxCoordinates} @@ -35,7 +35,6 @@ import org.json4s.JsonAST._ import scodec.bits.ByteVector import java.util.Base64 -import scala.concurrent.duration._ import scala.concurrent.{ExecutionContext, Future} import scala.jdk.CollectionConverters.ListHasAsScala import scala.util.{Failure, Success, Try} @@ -48,45 +47,20 @@ import scala.util.{Failure, Success, Try} * The Bitcoin Core client provides some high-level utility methods to interact with Bitcoin Core. * * @param rpcClient bitcoin core JSON rpc client - * @param onchainKeyManager_opt optional onchain key manager. If provided and its wallet name matches our rpcClient wallet's name, it will be used to sign transactions (it is assumed that bitcoin - * core uses a watch-only wallet with descriptors generated by Eclair with this onchain key manager) + * @param onChainKeyManager_opt optional on-chain key manager. If provided it will be used to sign transactions (it is assumed that bitcoin + * core uses a watch-only wallet with descriptors generated by Eclair with this on-chain key manager) */ -class BitcoinCoreClient(val rpcClient: BitcoinJsonRPCClient, val onchainKeyManager_opt: Option[OnchainKeyManager] = None) extends OnChainWallet with Logging { +class BitcoinCoreClient(val rpcClient: BitcoinJsonRPCClient, val onChainKeyManager_opt: Option[OnChainKeyManager] = None) extends OnChainWallet with Logging { import BitcoinCoreClient._ implicit val formats: Formats = org.json4s.DefaultFormats - val useEclairSigner = onchainKeyManager_opt.exists(m => rpcClient.wallet.contains(m.wallet)) - - //------------------------- WALLET -------------------------// - def importDescriptors(descriptors: Seq[Descriptor])(implicit ec: ExecutionContext): Future[Boolean] = { - rpcClient.invoke("importdescriptors", descriptors).collect { - case JArray(results) => results.forall(item => { - val JBool(success) = item \ "success" - success - }) - } + onChainKeyManager_opt.foreach { keyManager => + require(rpcClient.wallet.contains(keyManager.wallet), s"eclair-backed bitcoin wallet mismatch: eclair-signer.conf uses wallet=${keyManager.wallet}, but eclair.conf uses wallet=${rpcClient.wallet.getOrElse("")}") } - def createEclairBackedWallet()(implicit ec: ExecutionContext): Future[Boolean] = { - onchainKeyManager_opt match { - case None => Future.successful(true) // no eclair-backed wallet is configured - case Some(onchainKeyManager) if !rpcClient.wallet.contains(onchainKeyManager.wallet) => Future.successful(true) // configured wallet has a different name - case Some(onchainKeyManager) if onchainKeyManager.getWalletTimestamp() < (TimestampSecond.now() - 2.hours) => - logger.warn(s"descriptors are too old, you will need to manually import them and select how far back to rescan") - Future.failed(new RuntimeException("Could not import descriptors, please check logs for details")) - case Some(onchainKeyManager) => - logger.info(s"Creating a new on-chain eclair-backed wallet in bitcoind: ${onchainKeyManager.wallet}") - for { - // arguments to createwallet: wallet_name, disable_private_keys=true, blank=true, passphrase="", avoid_reuse=false, descriptors=true, load_on_startup=true - // we create an empty watch-only wallet, and then import our public key descriptors - _ <- rpcClient.invoke("createwallet", onchainKeyManager.wallet, true, true, "", false, true, true) - _ = logger.info(s"importing new descriptors ${onchainKeyManager.getDescriptors(0).descriptors}") - result <- importDescriptors(onchainKeyManager.getDescriptors(0).descriptors) - } yield result - } - } + val useEclairSigner = onChainKeyManager_opt.nonEmpty //------------------------- TRANSACTIONS -------------------------// @@ -259,6 +233,7 @@ class BitcoinCoreClient(val rpcClient: BitcoinJsonRPCClient, val onchainKeyManag } //------------------------- FUNDING -------------------------// + def fundTransaction(tx: Transaction, options: FundTransactionOptions)(implicit ec: ExecutionContext): Future[FundTransactionResponse] = { rpcClient.invoke("fundrawtransaction", tx.toString(), options).flatMap(json => { val JString(hex) = json \ "hex" @@ -272,12 +247,10 @@ class BitcoinCoreClient(val rpcClient: BitcoinJsonRPCClient, val onchainKeyManag Try { require(addedOutputs <= 1, "more than one change output added") require(addedOutputs == 0 || changePos >= 0, "change output added, but position not returned") - require(changePos < 0 || !tx.txOut.contains(fundedTx.txOut(changePos.intValue)), "existing output returned as change output") - require(options.changePosition.isEmpty || changePos == options.changePosition.get || changePos == -1, "change position added at wrong position") + require(options.changePosition.isEmpty || changePos_opt.isEmpty || changePos_opt == options.changePosition, "change output added at wrong position") FundTransactionResponse(fundedTx, toSatoshi(fee), changePos_opt) - } - match { + } match { case Success(response) => Future.successful(response) case Failure(error) => unlockOutpoints(walletInputs.toSeq).flatMap(_ => Future.failed(error)) } @@ -288,76 +261,72 @@ class BitcoinCoreClient(val rpcClient: BitcoinJsonRPCClient, val onchainKeyManag fundTransaction(tx, FundTransactionOptions(feeRate, replaceable, inputWeights = externalInputsWeight.map { case (outpoint, weight) => InputWeight(outpoint, weight) }.toSeq)) } - def processPsbt(psbt: Psbt, sign: Boolean = true, sighashType: Option[Int] = None)(implicit ec: ExecutionContext): Future[ProcessPsbtResponse] = { - // we use an explicit map here because this RPC calls takes a String for the "sighashtype" parameter with an explicitly limited list of valid values - val sighashStrings = Map( - SigHash.SIGHASH_DEFAULT -> "DEFAULT", - SigHash.SIGHASH_ALL -> "ALL", - SigHash.SIGHASH_NONE -> "NONE", - SigHash.SIGHASH_SINGLE -> "SINGLE", - (SigHash.SIGHASH_ALL | SigHash.SIGHASH_ANYONECANPAY) -> "ALL|ANYONECANPAY", - (SigHash.SIGHASH_NONE | SigHash.SIGHASH_ANYONECANPAY) -> "NONE|ANYONECANPAY", - (SigHash.SIGHASH_SINGLE | SigHash.SIGHASH_ANYONECANPAY) -> "SINGLE|ANYONECANPAY") - val sighash = sighashType.map(s => sighashStrings.getOrElse(s, throw new IllegalArgumentException(s"invalid sighash flag ${sighashType}"))) + private def processPsbt(psbt: Psbt, sign: Boolean = true, sighashType: Option[Int] = None)(implicit ec: ExecutionContext): Future[ProcessPsbtResponse] = { + // This RPC call takes a string for the "sighashtype" parameter with an explicitly limited list of valid values. + val sighash = sighashType match { + case None => None + case Some(sighash) if sighash == SigHash.SIGHASH_DEFAULT => Some("DEFAULT") + case Some(sighash) if sighash == SigHash.SIGHASH_ALL => Some("ALL") + case Some(sighash) if sighash == SigHash.SIGHASH_NONE => Some("NONE") + case Some(sighash) if sighash == SigHash.SIGHASH_SINGLE => Some("SINGLE") + case Some(sighash) if sighash == (SigHash.SIGHASH_ALL | SigHash.SIGHASH_ANYONECANPAY) => Some("ALL|ANYONECANPAY") + case Some(sighash) if sighash == (SigHash.SIGHASH_NONE | SigHash.SIGHASH_ANYONECANPAY) => Some("NONE|ANYONECANPAY") + case Some(sighash) if sighash == (SigHash.SIGHASH_SINGLE | SigHash.SIGHASH_ANYONECANPAY) => Some("SINGLE|ANYONECANPAY") + case _ => return Future.failed(new IllegalArgumentException(s"invalid sighash flag $sighashType")) + } val encoded = Base64.getEncoder.encodeToString(Psbt.write(psbt).toByteArray) val params = Seq(encoded, sign) ++ sighash.toSeq rpcClient.invoke("walletprocesspsbt", params: _*).map(json => { val JString(base64) = json \ "psbt" val JBool(complete) = json \ "complete" val decoded = Psbt.read(Base64.getDecoder.decode(base64)) - require(decoded.isRight, s"cannot decode psbt from $base64") + require(decoded.isRight, s"cannot decode processed psbt=$base64") ProcessPsbtResponse(decoded.getRight, complete) }) } - def utxoUpdatePsbt(psbt: Psbt)(implicit ec: ExecutionContext): Future[Psbt] = { + private def utxoUpdatePsbt(psbt: Psbt)(implicit ec: ExecutionContext): Future[Psbt] = { val encoded = Base64.getEncoder.encodeToString(Psbt.write(psbt).toByteArray) - rpcClient.invoke("utxoupdatepsbt", encoded).map(json => { val JString(base64) = json - val bin = Base64.getDecoder.decode(base64) - val decoded = Psbt.read(bin) - require(decoded.isRight, s"cannot decode psbt from $base64") - val psbt = decoded.getRight - psbt + val decoded = Psbt.read(Base64.getDecoder.decode(base64)) + require(decoded.isRight, s"cannot decode updated psbt=$base64") + decoded.getRight }) } - private def unlockIfFails[T](txid: ByteVector32, locked: Seq[OutPoint])(f: => Future[T])(implicit ec: ExecutionContext): Future[T] = { - // if signature fails (e.g. because wallet is encrypted) we need to unlock the utxos - f.recoverWith { case _ => - unlockOutpoints(locked) - .recover { case t: Throwable => // no-op, just add a log in case of failure - logger.warn(s"Cannot unlock failed transaction's UTXOs txid=${txid}", t) - t - } - .flatMap(_ => f) // return signTransaction error - .recoverWith { case _ => f } // return signTransaction error + private def unlockIfFails[T](locked: Seq[OutPoint])(f: => Future[T])(implicit ec: ExecutionContext): Future[T] = { + f.transformWith { + // We preserve the original failure, regardless of the result of the utxo unlocking call. + case Failure(f) => unlockOutpoints(locked).transformWith { _ => Future.failed(f) } + case Success(result) => Future.successful(result) } } def makeFundingTx(pubkeyScript: ByteVector, amount: Satoshi, targetFeerate: FeeratePerKw)(implicit ec: ExecutionContext): Future[MakeFundingTxResponse] = { - import KotlinUtils._ - def sign(tx: Transaction, fees: Satoshi, requestedFeeRate: FeeratePerKw): Future[MakeFundingTxResponse] = { - val outputIndex = Transactions.findPubKeyScriptIndex(tx, pubkeyScript) getOrElse { - throw new RuntimeException(s"cannot find expected funding output") + def verifyAndSign(tx: Transaction, fees: Satoshi, requestedFeeRate: FeeratePerKw): Future[MakeFundingTxResponse] = { + import KotlinUtils._ + + val fundingOutputIndex = Transactions.findPubKeyScriptIndex(tx, pubkeyScript) match { + case Left(_) => return Future.failed(new RuntimeException("cannot find expected funding output: bitcoin core may be malicious")) + case Right(outputIndex) => outputIndex } - val ourInputs = tx.txIn.indices.toList // all inputs are supposed to be ours - val ourOutputs = tx.txOut.indices.toList filterNot (_ == outputIndex) // all outputs except for the funding output are supposed to be ours + val ourInputs = tx.txIn.indices.toList // all inputs are supposed to belong to our bitcoin wallet + val ourOutputs = tx.txOut.indices.toList.filterNot(_ == fundingOutputIndex) // all outputs except for the funding output are supposed to belong to our bitcoin wallet val psbt = new Psbt(tx) for { signed <- signPsbt(psbt, ourInputs, ourOutputs) extracted = signed.psbt.extract() _ = require(extracted.isRight, s"signing psbt failed with ${extracted.getLeft}") actualFees = kmp2scala(signed.psbt.computeFees()) - _ = require(actualFees == fees, s"actual funding fees $actualFees do not match bitcoin core fee $fees") + _ = require(actualFees == fees, s"actual funding fees $actualFees do not match returned fees $fees: bitcoin core may be malicious") fundingTx = kmp2scala(extracted.getRight) actualFeerate = Transactions.fee2rate(actualFees, fundingTx.weight()) - maxFeerate = requestedFeeRate + requestedFeeRate / 2 - _ = require(actualFeerate < maxFeerate, s"actual feerate $actualFeerate is more than 50% above requested fee rate $targetFeerate") - _ = logger.debug(s"created funding txid=${extracted.getRight.txid} outputIndex=$outputIndex fee=${fees}") - } yield MakeFundingTxResponse(fundingTx, outputIndex, fees) + maxFeerate = requestedFeeRate * 1.5 + _ = require(actualFeerate < maxFeerate, s"actual feerate $actualFeerate is more than 50% above requested feerate $targetFeerate") + _ = logger.debug(s"created funding txid=${fundingTx.txid} outputIndex=$fundingOutputIndex fee=$fees") + } yield MakeFundingTxResponse(fundingTx, fundingOutputIndex, fees) } val partialFundingTx = Transaction( @@ -372,7 +341,7 @@ class BitcoinCoreClient(val rpcClient: BitcoinJsonRPCClient, val onchainKeyManag // we ask bitcoin core to add inputs to the funding tx, and use the specified change address FundTransactionResponse(tx, fee, _) <- fundTransaction(partialFundingTx, FundTransactionOptions(feerate)) lockedUtxos = tx.txIn.map(_.outPoint) - signedTx <- unlockIfFails(tx.txid, lockedUtxos)(sign(tx, fee, feerate)) + signedTx <- unlockIfFails(lockedUtxos)(verifyAndSign(tx, fee, feerate)) } yield signedTx } @@ -425,12 +394,11 @@ class BitcoinCoreClient(val rpcClient: BitcoinJsonRPCClient, val onchainKeyManag } else { val unsignedTx = Transaction(2, outpoints.toSeq.map(o => TxIn(o, Seq.empty, 0)), Seq(TxOut(amountIn - missingFees, Script.pay2wpkh(changePubkeyHash))), 0) signPsbt(new Psbt(unsignedTx), unsignedTx.txIn.indices, unsignedTx.txOut.indices).transformWith { - case Failure(ex) => Future.failed(new IllegalArgumentException("tx signing failed: some inputs don't belong to our wallet", ex)) - case Success(response) => - response.extractFinalTx match { - case Left(error) => Future.failed(new IllegalArgumentException("tx signing failed: some inputs don't belong to our wallet", new RuntimeException(error.toString))) - case Right(tx) => publishTransaction(tx).map(_ => tx) - } + case Failure(ex) => Future.failed(new IllegalArgumentException("tx signing failed", ex)) + case Success(response) => response.finalTx_opt match { + case Left(error) => Future.failed(new IllegalArgumentException(s"tx signing failed: ${error.toString}")) + case Right(tx) => publishTransaction(tx).map(_ => tx) + } } } } @@ -470,22 +438,22 @@ class BitcoinCoreClient(val rpcClient: BitcoinJsonRPCClient, val onchainKeyManag //------------------------- SIGNING -------------------------// def signPsbt(psbt: Psbt, ourInputs: Seq[Int], ourOutputs: Seq[Int])(implicit ec: ExecutionContext): Future[ProcessPsbtResponse] = { - - def sign(psbt: Psbt): Future[ProcessPsbtResponse] = if (useEclairSigner) { - val onchainKeyManager = onchainKeyManager_opt.getOrElse(throw new RuntimeException("We are configured to use an eclair signer has not been loaded")) // this should not be possible - onchainKeyManager.signPsbt(psbt, ourInputs, ourOutputs) match { - case Success(signedPsbt) => Future.successful(ProcessPsbtResponse(signedPsbt, signedPsbt.extract().isRight)) - case Failure(error) => Future.failed(error) - } - } else { - processPsbt(psbt, sign = true) + onChainKeyManager_opt match { + case Some(keyManager) => + for { + updated <- utxoUpdatePsbt(psbt) + filled <- processPsbt(updated, sign = false) // just fill input and output HD paths + signed <- keyManager.sign(filled.psbt, ourInputs, ourOutputs) match { + case Success(signedPsbt) => Future.successful(ProcessPsbtResponse(signedPsbt, signedPsbt.extract().isRight)) + case Failure(error) => Future.failed(error) + } + } yield signed + case None => + for { + updated <- utxoUpdatePsbt(psbt) + signed <- processPsbt(updated, sign = true) + } yield signed } - - for { - updated <- utxoUpdatePsbt(psbt) - filled <- processPsbt(updated, sign = false) // called with sign=false to fill input and output HD paths - signed <- sign(filled.psbt) - } yield signed } //------------------------- PUBLISHING -------------------------// @@ -572,11 +540,13 @@ class BitcoinCoreClient(val rpcClient: BitcoinJsonRPCClient, val onchainKeyManag val extracted = PublicKey(ByteVector.fromValidHex(rawKey)) // check that when we manage private keys we can re-compute the public key we got from bitcoin core // and that the address and public key match - if (useEclairSigner) { - val onchainKeyManager = onchainKeyManager_opt.getOrElse(throw new RuntimeException("We are configured to use an eclair signer has not been loaded")) // this should not be possible - val keyPath1 = keyPath.replace('h', '\'') // our bitcoin lib expects a ' suffix for hardened indexes and does not yet accept the h suffix - val computed = onchainKeyManager.getPublicKey(DeterministicWallet.KeyPath(keyPath1)) - require(computed == (extracted, address), "cannot recompute pubkey generated by bitcoin core") + onChainKeyManager_opt match { + case Some(keyManager) => + // TODO: bitcoin-kmp doesn't accept 'h' for hardened indexes, we should fix this. + val keyPath1 = keyPath.replace('h', '\'') + val computed = keyManager.derivePublicKey(DeterministicWallet.KeyPath(keyPath1)) + if (computed != (extracted, address)) return Future.failed(new RuntimeException("cannot recompute pubkey generated by bitcoin core")) + case None => () } extracted } @@ -592,21 +562,16 @@ class BitcoinCoreClient(val rpcClient: BitcoinJsonRPCClient, val onchainKeyManag pubKey <- extractPublicKey(address) } yield pubKey - /** - * @return the public key hash of a bech32 raw change address. - */ - def getP2wpkhPubkeyHashForChange()(implicit ec: ExecutionContext): Future[ByteVector] = { - rpcClient.invoke("getrawchangeaddress", "bech32").collect { - case JString(changeAddress) => - val pubkeyHash = ByteVector.view(Bech32.decodeWitnessAddress(changeAddress).getThird) - pubkeyHash - } - } - + /** @return the public key hash of a bech32 raw change address. */ + def getP2wpkhPubkeyHashForChange()(implicit ec: ExecutionContext): Future[ByteVector] = for { + JString(changeAddress) <- rpcClient.invoke("getrawchangeaddress", "bech32") + _ <- extractPublicKey(changeAddress) + pubkeyHash = ByteVector.view(Bech32.decodeWitnessAddress(changeAddress).getThird) + } yield pubkeyHash /** - * Asks Bitcoin Core to fund and broadcsat a tx that sends funds to a given pubkey script - * If the current wallet uses Eclair to sign transaction, then we'll use our onchain key manager to sign the transaction, + * Ask Bitcoin Core to fund and broadcast a tx that sends funds to a given pubkey script. + * If the current wallet uses Eclair to sign transaction, then we'll use our on-chain key manager to sign the transaction, * with the following assumptions: * - all inputs belong to us * - all outputs except for the one that sends to `pubkeyScript` belong to us @@ -622,22 +587,23 @@ class BitcoinCoreClient(val rpcClient: BitcoinJsonRPCClient, val onchainKeyManag val theirOutput = TxOut(amount, pubkeyScript) val tx = Transaction(version = 2, txIn = Nil, txOut = theirOutput :: Nil, lockTime = 0) for { - fundedTx <- fundTransaction(tx, feeratePerKw, true) + fundedTx <- fundTransaction(tx, feeratePerKw, replaceable = true) lockedOutputs = fundedTx.tx.txIn.map(_.outPoint) theirOutputPos = fundedTx.tx.txOut.indexOf(theirOutput) - signedPsbt <- unlockIfFails(fundedTx.tx.txid, lockedOutputs)(signPsbt(new Psbt(fundedTx.tx), fundedTx.tx.txIn.indices, fundedTx.tx.txOut.indices.filterNot(_ == theirOutputPos))) - signedTx = signedPsbt.finalTx + signedPsbt <- unlockIfFails(lockedOutputs)(signPsbt(new Psbt(fundedTx.tx), fundedTx.tx.txIn.indices, fundedTx.tx.txOut.indices.filterNot(_ == theirOutputPos))) + _ = require(signedPsbt.finalTx_opt.isRight, s"transaction was not fully signed (${signedPsbt.finalTx_opt.left.toOption.get}): bitcoin core may be malicious") + signedTx = signedPsbt.finalTx_opt.toOption.get actualFees = kmp2scala(signedPsbt.psbt.computeFees()) actualFeerate = Transactions.fee2rate(actualFees, signedTx.weight()) - maxFeerate = feeratePerKw + feeratePerKw / 2 - _ = require(actualFeerate < maxFeerate, s"actual fee rate $actualFeerate is more than 50% above requested fee rate $feeratePerKw") + maxFeerate = feeratePerKw * 1.5 + _ = require(actualFeerate < maxFeerate, s"actual feerate $actualFeerate is more than 50% above requested feerate $feeratePerKw") txid <- publishTransaction(signedTx) } yield txid } def sendToPubkeyScript(pubkeyScript: Seq[ScriptElt], amount: Satoshi, feeratePerKw: FeeratePerKw)(implicit ec: ExecutionContext): Future[ByteVector32] = sendToPubkeyScript(Script.write(pubkeyScript), amount, feeratePerKw) - // calls Bitcoin Core's sendtoaddress RPC call directly. Will fail if wallet is using an external signer + /** Calls Bitcoin Core's sendtoaddress RPC call directly. Will fail if wallet is using an external signer. */ def sendToAddress(address: String, amount: Satoshi, confirmationTarget: Long)(implicit ec: ExecutionContext): Future[ByteVector32] = { rpcClient.invoke( "sendtoaddress", @@ -762,45 +728,6 @@ object BitcoinCoreClient { } } - case class ProcessPsbtResponse(psbt: Psbt, complete: Boolean) { - - import KotlinUtils._ - - // Extract a fully signed transaction from `psbt` - // If the transaction is just partially signed, this method will fail and you must call extractPartiallySignedTx instead - def extractFinalTx: Either[UpdateFailure, Transaction] = { - val extracted = psbt.extract() - if (extracted.isLeft) Left(extracted.getLeft) else Right(extracted.getRight) - } - - // Extract a partially signed transaction from `psbt` - def extractPartiallySignedTx: Transaction = { - var partiallySignedTx: Transaction = psbt.getGlobal.getTx - for (i <- 0 until psbt.getInputs.size()) { - val scriptWitness = psbt.getInputs.get(i).getScriptWitness - if (scriptWitness != null) { - partiallySignedTx = partiallySignedTx.updateWitness(i, scriptWitness) - } - } - partiallySignedTx - } - - def finalTx = extractFinalTx.getOrElse(throw new RuntimeException("cannot extract transaction from psbt")) - } - - case class PreviousTx(txid: ByteVector32, vout: Long, scriptPubKey: String, redeemScript: String, witnessScript: String, amount: BigDecimal) - - object PreviousTx { - def apply(inputInfo: Transactions.InputInfo, witness: ScriptWitness): PreviousTx = PreviousTx( - inputInfo.outPoint.txid, - inputInfo.outPoint.index, - inputInfo.txOut.publicKeyScript.toHex, - inputInfo.redeemScript.toHex, - ScriptWitness.write(witness).toHex, - inputInfo.txOut.amount.toBtc.toBigDecimal - ) - } - /** * Information about a transaction currently in the mempool. * diff --git a/eclair-core/src/main/scala/fr/acinq/eclair/channel/fund/InteractiveTxBuilder.scala b/eclair-core/src/main/scala/fr/acinq/eclair/channel/fund/InteractiveTxBuilder.scala index 8e4d216833..998f8e5d11 100644 --- a/eclair-core/src/main/scala/fr/acinq/eclair/channel/fund/InteractiveTxBuilder.scala +++ b/eclair-core/src/main/scala/fr/acinq/eclair/channel/fund/InteractiveTxBuilder.scala @@ -773,28 +773,30 @@ private class InteractiveTxBuilder(replyTo: ActorRef[InteractiveTxBuilder.Respon if (unsignedTx.localInputs.isEmpty) { context.self ! SignTransactionResult(PartiallySignedSharedTransaction(unsignedTx, TxSignatures(fundingParams.channelId, tx, Nil, sharedSig_opt))) } else { - // we only sign our wallet inputs, and check that we can spend our wallet outputs - val ourWalletInputs = unsignedTx.localInputs.map(i => tx.txIn.indexWhere(_.outPoint == OutPoint(i.previousTx, i.previousTxOutput.toInt))) - val ourWalletOutputs = unsignedTx.localOutputs.collect { - // README!! there are 2 types of local outputs: - // Change, which go back into our wallet - // NonChange, which go to an external address (typically during a splice-out) - // Here we only keep outputs which are ours i.e go back into our wallet - // And we trust that NonChange outputs are valid. This only works if the entry point for creating such outputs is trusted (for example, a secure API call) - case Output.Local.Change(_, amount, pubkeyScript) => tx.txOut.indexWhere(output => output.amount == amount && output.publicKeyScript == pubkeyScript) + // We only sign our wallet inputs, and check that we can spend our wallet outputs. + val ourWalletInputs = unsignedTx.localInputs.map(i => tx.txIn.indexWhere(_.outPoint == i.outPoint)) + val ourWalletOutputs = unsignedTx.localOutputs.flatMap { + case Output.Local.Change(_, amount, pubkeyScript) => Some(tx.txOut.indexWhere(output => output.amount == amount && output.publicKeyScript == pubkeyScript)) + // Non-change outputs may go to an external address (typically during a splice-out). + // Here we only keep outputs which are ours i.e explicitly go back into our wallet. + // We trust that non-change outputs are valid: this only works if the entry point for creating such outputs is trusted (for example, a secure API call). + case _: Output.Local.NonChange => None } context.pipeToSelf(wallet.signPsbt(new Psbt(tx), ourWalletInputs, ourWalletOutputs).map { response => val localOutpoints = unsignedTx.localInputs.map(_.outPoint).toSet - val partiallySignedTx = response.extractPartiallySignedTx - // partially signed PSBT must include spent amounts for all inputs that were signed, and we can "trust" these amounts because they are included - // in the hash that we signed (see BIP143). If our bitcoin node lied about them, then our signatures are invalid + val partiallySignedTx = response.partiallySignedTx + // Partially signed PSBT must include spent amounts for all inputs that were signed, and we can "trust" these amounts because they are included + // in the hash that we signed (see BIP143). If our bitcoin node lied about them, then our signatures are invalid. val actualLocalAmountIn = ourWalletInputs.map(i => kmp2scala(response.psbt.getInput(i).getWitnessUtxo.amount)).sum val expectedLocalAmountIn = unsignedTx.localInputs.map(i => i.previousTx.txOut(i.previousTxOutput.toInt).amount).sum - require(actualLocalAmountIn == expectedLocalAmountIn, s"local spent amount ${actualLocalAmountIn} does not match what we expect ($expectedLocalAmountIn") + require(actualLocalAmountIn == expectedLocalAmountIn, s"local spent amount $actualLocalAmountIn does not match what we expect ($expectedLocalAmountIn): bitcoin core may be malicious") val actualLocalAmountOut = ourWalletOutputs.map(i => partiallySignedTx.txOut(i).amount).sum - val expectedLocalAmountOut = unsignedTx.localOutputs.collect { case c: Output.Local.Change => c.amount }.sum - require(actualLocalAmountOut == expectedLocalAmountOut, s"local output amount ${actualLocalAmountOut} does not match what we expect ($expectedLocalAmountOut") + val expectedLocalAmountOut = unsignedTx.localOutputs.map { + case c: Output.Local.Change => c.amount + case _: Output.Local.NonChange => 0.sat + }.sum + require(actualLocalAmountOut == expectedLocalAmountOut, s"local output amount $actualLocalAmountOut does not match what we expect ($expectedLocalAmountOut): bitcoin core may be malicious") val sigs = partiallySignedTx.txIn.filter(txIn => localOutpoints.contains(txIn.outPoint)).map(_.witness) PartiallySignedSharedTransaction(unsignedTx, TxSignatures(fundingParams.channelId, partiallySignedTx, sigs, sharedSig_opt)) }) { diff --git a/eclair-core/src/main/scala/fr/acinq/eclair/channel/publish/ReplaceableTxFunder.scala b/eclair-core/src/main/scala/fr/acinq/eclair/channel/publish/ReplaceableTxFunder.scala index 160fdb2d1b..d5f8957e35 100644 --- a/eclair-core/src/main/scala/fr/acinq/eclair/channel/publish/ReplaceableTxFunder.scala +++ b/eclair-core/src/main/scala/fr/acinq/eclair/channel/publish/ReplaceableTxFunder.scala @@ -19,7 +19,7 @@ package fr.acinq.eclair.channel.publish import akka.actor.typed.eventstream.EventStream import akka.actor.typed.scaladsl.{ActorContext, Behaviors} import akka.actor.typed.{ActorRef, Behavior} -import fr.acinq.bitcoin.psbt.{Psbt, UpdateFailure} +import fr.acinq.bitcoin.psbt.Psbt import fr.acinq.bitcoin.scalacompat.{ByteVector32, OutPoint, Satoshi, Script, Transaction, TxOut} import fr.acinq.eclair.NotificationsLogger.NotifyNodeOperator import fr.acinq.eclair.blockchain.OnChainWallet @@ -295,7 +295,7 @@ private class ReplaceableTxFunder(nodeParams: NodeParams, } Behaviors.receiveMessagePartial { case AddInputsOk(fundedTx, totalAmountIn) => - log.info("added {} wallet input(s) and {} wallet output(s) to {}", fundedTx.txInfo.tx.txIn.length - 1, fundedTx.txInfo.tx.txOut.length - 1, cmd.desc) + log.debug("added {} wallet input(s) and {} wallet output(s) to {}", fundedTx.txInfo.tx.txIn.length - 1, fundedTx.txInfo.tx.txOut.length - 1, cmd.desc) sign(fundedTx, targetFeerate, totalAmountIn) case AddInputsFailed(reason) => if (reason.getMessage.contains("Insufficient funds")) { @@ -355,41 +355,39 @@ private class ReplaceableTxFunder(nodeParams: NodeParams, private def signWalletInputs(locallySignedTx: ReplaceableTxWithWalletInputs, txFeerate: FeeratePerKw, amountIn: Satoshi): Behavior[Command] = { import fr.acinq.bitcoin.scalacompat.KotlinUtils._ - // we finalize (sign) the input that we control, and will then ask our bitcoin client to sign wallet inputs + // We create a PSBT with the non-wallet input already signed: val psbt = new Psbt(locallySignedTx.txInfo.tx) - val updated: Either[UpdateFailure, Psbt] = psbt.updateWitnessInput(locallySignedTx.txInfo.input.outPoint, locallySignedTx.txInfo.input.txOut, null, fr.acinq.bitcoin.Script.parse(locallySignedTx.txInfo.input.redeemScript), null, java.util.Map.of()) - val finalized = updated.flatMap(_.finalizeWitnessInput(0, locallySignedTx.txInfo.tx.txIn.head.witness)) - finalized match { + .updateWitnessInput(locallySignedTx.txInfo.input.outPoint, locallySignedTx.txInfo.input.txOut, null, fr.acinq.bitcoin.Script.parse(locallySignedTx.txInfo.input.redeemScript), fr.acinq.bitcoin.SigHash.SIGHASH_ALL, java.util.Map.of()) + .flatMap(_.finalizeWitnessInput(0, locallySignedTx.txInfo.tx.txIn.head.witness)) + psbt match { case Left(failure) => - log.error(s"cannot sign ${cmd.desc}: ", failure) + log.error(s"cannot sign ${cmd.desc}: $failure") unlockAndStop(locallySignedTx.txInfo.input.outPoint, locallySignedTx.txInfo.tx, TxPublisher.TxRejectedReason.UnknownTxFailure) case Right(psbt1) => - // the transaction that we want to fund/replace has one input, the first one. Additional inputs are provided by our onchain wallet. - val ourWalletInputs = locallySignedTx.txInfo.tx.txIn.indices.drop(1) - // for "claim anchor txs" there is a single change output that sends to our onchain wallet - // for htlc txs the first output is the one we want to fund/bump, additional outputs send to our onchain wallet + // The transaction that we want to fund/replace has one input, the first one. Additional inputs are provided by our on-chain wallet. + val ourWalletInputs = locallySignedTx.txInfo.tx.txIn.indices.tail + // For "claim anchor txs" there is a single change output that sends to our on-chain wallet. + // For htlc txs the first output is the one we want to fund/bump, additional outputs send to our on-chain wallet. val ourWalletOutputs = locallySignedTx match { case _: ClaimLocalAnchorWithWitnessData => Seq(0) - case _: HtlcWithWitnessData => locallySignedTx.txInfo.tx.txOut.indices.drop(1) + case _: HtlcWithWitnessData => locallySignedTx.txInfo.tx.txOut.indices.tail } context.pipeToSelf(bitcoinClient.signPsbt(psbt1, ourWalletInputs, ourWalletOutputs)) { case Success(processPsbtResponse) => - val signedTx = processPsbtResponse.finalTx - val actualFees = kmp2scala(processPsbtResponse.psbt.computeFees()) - val actualWeight = locallySignedTx match { - case _: ClaimLocalAnchorWithWitnessData => signedTx.weight() + dummySignedCommitTx(cmd.commitment).tx.weight() - case _ => - locallySignedTx.txInfo match { - case _: HtlcSuccessTx => cmd.commitment.params.commitmentFormat.htlcSuccessInputWeight + signedTx.weight() - case _: HtlcTimeoutTx => cmd.commitment.params.commitmentFormat.htlcTimeoutInputWeight + signedTx.weight() + processPsbtResponse.finalTx_opt match { + case Right(signedTx) => + val actualFees = kmp2scala(processPsbtResponse.psbt.computeFees()) + val actualWeight = locallySignedTx match { + case _: ClaimLocalAnchorWithWitnessData => signedTx.weight() + dummySignedCommitTx(cmd.commitment).tx.weight() case _ => signedTx.weight() } - } - val actualFeerate = Transactions.fee2rate(actualFees, actualWeight) - if (actualFeerate >= txFeerate * 2) { - SignWalletInputsFailed(new RuntimeException(s"actual fee rate $actualFeerate is more than twice the requested fee rate $txFeerate")) - } else { - SignWalletInputsOk(signedTx) + val actualFeerate = Transactions.fee2rate(actualFees, actualWeight) + if (actualFeerate >= txFeerate * 2) { + SignWalletInputsFailed(new RuntimeException(s"actual fee rate $actualFeerate is more than twice the requested fee rate $txFeerate")) + } else { + SignWalletInputsOk(signedTx) + } + case Left(failure) => SignWalletInputsFailed(new RuntimeException(s"could not sign psbt: $failure")) } case Failure(reason) => SignWalletInputsFailed(reason) } @@ -439,31 +437,32 @@ private class ReplaceableTxFunder(nodeParams: NodeParams, // The anchor transaction is paying for the weight of the commitment transaction. val anchorWeight = Seq(InputWeight(anchorTx.txInfo.input.outPoint, anchorInputWeight + commitTx.weight())) - - def makeSingleOutputTx(fundTxResponse: OnChainWallet.FundTransactionResponse): Future[Transaction] = fundTxResponse.changePosition match { - case Some(changePos) => - val changeOutput = fundTxResponse.tx.txOut(changePos).copy(amount = fundTxResponse.tx.txOut.map(_.amount).sum) - val txSingleOutput = fundTxResponse.tx.copy(txOut = Seq(changeOutput)) - Future.successful(txSingleOutput) - case None => - bitcoinClient.getP2wpkhPubkeyHashForChange().map(pubkeyHash => { - // replace PlaceHolderPubKey with a real wallet key - val fundedTx = fundTxResponse.tx.copy(txOut = Seq(TxOut(dustLimit, Script.pay2wpkh(pubkeyHash)))) - fundedTx - }) + def makeSingleOutputTx(fundTxResponse: OnChainWallet.FundTransactionResponse): Future[Transaction] = { + // Bitcoin Core may not preserve the order of inputs, we need to make sure the anchor is the first input. + val txIn = anchorTx.txInfo.tx.txIn ++ fundTxResponse.tx.txIn.filterNot(_.outPoint == anchorTx.txInfo.input.outPoint) + fundTxResponse.changePosition match { + case Some(changePos) => + val changeOutput = fundTxResponse.tx.txOut(changePos).copy(amount = fundTxResponse.tx.txOut.map(_.amount).sum) + val txSingleOutput = fundTxResponse.tx.copy(txIn = txIn, txOut = Seq(changeOutput)) + Future.successful(txSingleOutput) + case None => + bitcoinClient.getP2wpkhPubkeyHashForChange().map(pubkeyHash => { + // replace PlaceHolderPubKey with a real wallet key + fundTxResponse.tx.copy(txIn = txIn, txOut = Seq(TxOut(dustLimit, Script.pay2wpkh(pubkeyHash)))) + }) + } } for { fundTxResponse <- bitcoinClient.fundTransaction(txNotFunded, FundTransactionOptions(targetFeerate, inputWeights = anchorWeight)) txSingleOutput <- makeSingleOutputTx(fundTxResponse) - changeOutput = txSingleOutput.txOut(0) - ourWalletInputs = fundTxResponse.tx.txIn.indices.filterNot(i => fundTxResponse.tx.txIn(i).outPoint == anchorTx.txInfo.input.outPoint) + changeOutput = txSingleOutput.txOut.head + ourWalletInputs = txSingleOutput.txIn.indices.tail // all inputs except the first one ourWalletOutputs = Seq(0) // one change output // We ask bitcoind to sign the wallet inputs to learn their final weight and adjust the change amount. - psbt = new Psbt(txSingleOutput) - processPsbtResponse <- bitcoinClient.signPsbt(psbt, ourWalletInputs, ourWalletOutputs) + processPsbtResponse <- bitcoinClient.signPsbt(new Psbt(txSingleOutput), ourWalletInputs, ourWalletOutputs) // we cannot extract the final tx from the psbt because it is not fully signed yet - partiallySignedTx = processPsbtResponse.extractPartiallySignedTx + partiallySignedTx = processPsbtResponse.partiallySignedTx dummySignedTx = addSigs(anchorTx.updateTx(partiallySignedTx).txInfo, PlaceHolderSig) packageWeight = commitTx.weight() + dummySignedTx.tx.weight() // above, we asked bitcoin core to use the package weight to estimate fees when it built and funded this transaction, so we @@ -473,7 +472,7 @@ private class ReplaceableTxFunder(nodeParams: NodeParams, anchorTxFee = weight2fee(targetFeerate, packageWeight) - weight2fee(commitment.localCommit.spec.commitTxFeerate, commitTx.weight()) changeAmount = dustLimit.max(fundTxResponse.amountIn - anchorTxFee) - fundedTx = fundTxResponse.tx.copy(txOut = Seq(changeOutput.copy(amount = changeAmount))) + fundedTx = txSingleOutput.copy(txOut = Seq(changeOutput.copy(amount = changeAmount))) } yield { (anchorTx.updateTx(fundedTx), fundTxResponse.amountIn) } @@ -487,17 +486,17 @@ private class ReplaceableTxFunder(nodeParams: NodeParams, case _: HtlcTimeoutTx => commitment.params.commitmentFormat.htlcTimeoutInputWeight }) bitcoinClient.fundTransaction(htlcTx.txInfo.tx, FundTransactionOptions(targetFeerate, changePosition = Some(1), inputWeights = Seq(htlcInputWeight))).flatMap(fundTxResponse => { - val ourWalletInputs = fundTxResponse.tx.txIn.indices.filterNot(i => fundTxResponse.tx.txIn(i).outPoint == htlcTx.txInfo.input.outPoint) - val ourWalletOutputs = if (fundTxResponse.tx.txOut.size > 1) Seq(1) else Nil // there may not be a change output - val unsignedTx = htlcTx.updateTx(fundTxResponse.tx) - val psbt = new Psbt(fundTxResponse.tx) - bitcoinClient.signPsbt(psbt, ourWalletInputs, ourWalletOutputs).map(processPsbtResponse => { + // Bitcoin Core may not preserve the order of inputs, we need to make sure the htlc is the first input. + val fundedTx = fundTxResponse.tx.copy(txIn = htlcTx.txInfo.tx.txIn ++ fundTxResponse.tx.txIn.filterNot(_.outPoint == htlcTx.txInfo.input.outPoint)) + val ourWalletInputs = fundedTx.txIn.indices.tail // all inputs except the first one + val ourWalletOutputs = if (fundedTx.txOut.size > 1) Seq(1) else Nil // there may not be a change output + val unsignedTx = htlcTx.updateTx(fundedTx) + bitcoinClient.signPsbt(new Psbt(fundedTx), ourWalletInputs, ourWalletOutputs).map(processPsbtResponse => { val actualFees: Satoshi = processPsbtResponse.psbt.computeFees() - require(actualFees == fundTxResponse.fee, s"Bitcoin Core fees (${fundTxResponse.fee} do not match ours ($actualFees)") - val packageWeight = fundTxResponse.tx.weight() + htlcInputWeight.weight + require(actualFees == fundTxResponse.fee, s"Bitcoin Core fees (${fundTxResponse.fee}) do not match ours ($actualFees)") + val packageWeight = fundedTx.weight() + htlcInputWeight.weight val actualFeerate = Transactions.fee2rate(fundTxResponse.fee, packageWeight.toInt) require(actualFeerate < targetFeerate * 2, s"actual fee rate $actualFeerate is more than twice the requested fee rate $targetFeerate") - (unsignedTx, fundTxResponse.amountIn) }) }) diff --git a/eclair-core/src/main/scala/fr/acinq/eclair/crypto/keymanager/LocalOnChainKeyManager.scala b/eclair-core/src/main/scala/fr/acinq/eclair/crypto/keymanager/LocalOnChainKeyManager.scala new file mode 100644 index 0000000000..35cdeeed07 --- /dev/null +++ b/eclair-core/src/main/scala/fr/acinq/eclair/crypto/keymanager/LocalOnChainKeyManager.scala @@ -0,0 +1,239 @@ +/* + * Copyright 2023 ACINQ SAS + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package fr.acinq.eclair.crypto.keymanager + +import com.typesafe.config.ConfigFactory +import fr.acinq.bitcoin.psbt.{Psbt, UpdateFailure} +import fr.acinq.bitcoin.scalacompat.DeterministicWallet._ +import fr.acinq.bitcoin.scalacompat.{Block, ByteVector32, Crypto, DeterministicWallet, MnemonicCode, Satoshi, Script, computeBIP84Address} +import fr.acinq.eclair.TimestampSecond +import fr.acinq.eclair.blockchain.bitcoind.rpc.BitcoinCoreClient.{Descriptor, Descriptors} +import fr.acinq.eclair.blockchain.bitcoind.rpc.BitcoinJsonRPCClient +import grizzled.slf4j.Logging +import org.json4s.{JArray, JBool} +import scodec.bits.ByteVector + +import java.io.File +import scala.concurrent.duration.DurationInt +import scala.concurrent.{ExecutionContext, Future} +import scala.jdk.CollectionConverters.{CollectionHasAsScala, MapHasAsScala} +import scala.util.{Failure, Success, Try} + +object LocalOnChainKeyManager extends Logging { + def descriptorChecksum(span: String): String = fr.acinq.bitcoin.Descriptor.checksum(span) + + /** + * Load a configuration file and create an on-chain key manager + * + * @param datadir eclair data directory + * @param chainHash chain we're on + * @return a LocalOnChainKeyManager instance if a configuration file exists + */ + def load(datadir: File, chainHash: ByteVector32): Option[LocalOnChainKeyManager] = { + // we use a specific file instead of adding values to eclair's configuration file because it is available everywhere + // in the code through the actor system's settings and we'd like to restrict access to the on-chain wallet seed + val file = new File(datadir, "eclair-signer.conf") + if (file.exists()) { + val config = ConfigFactory.parseFile(file) + val wallet = config.getString("eclair.signer.wallet") + val mnemonics = config.getString("eclair.signer.mnemonics") + val passphrase = config.getString("eclair.signer.passphrase") + val timestamp = config.getLong("eclair.signer.timestamp") + val keyManager = new LocalOnChainKeyManager(wallet, MnemonicCode.toSeed(mnemonics, passphrase), TimestampSecond(timestamp), chainHash) + logger.info(s"using on-chain key manager wallet=$wallet xpub=${keyManager.masterPubKey(0)}") + Some(keyManager) + } else { + None + } + } +} + +class LocalOnChainKeyManager(override val wallet: String, seed: ByteVector, override val walletTimestamp: TimestampSecond, chainHash: ByteVector32) extends OnChainKeyManager with Logging { + + import LocalOnChainKeyManager._ + + // Master key derived from our seed. We use it to generate a BIP84 wallet that can be used: + // - to generate a watch-only wallet with any BIP84-compatible bitcoin wallet + // - to generate descriptors that can be imported into Bitcoin Core to create a watch-only wallet which can be used + // by Eclair to fund transactions (only Eclair will be able to sign wallet inputs) + private val master = DeterministicWallet.generate(seed) + private val fingerprint = DeterministicWallet.fingerprint(master) & 0xFFFFFFFFL + private val fingerPrintHex = String.format("%8s", fingerprint.toHexString).replace(' ', '0') + // Root BIP32 on-chain path: we use BIP84 (p2wpkh) paths: m/84'/{0'/1'} + private val rootPath = chainHash match { + case Block.RegtestGenesisBlock.hash | Block.TestnetGenesisBlock.hash | Block.SignetGenesisBlock.hash => "84'/1'" + case Block.LivenetGenesisBlock.hash => "84'/0'" + case _ => throw new IllegalArgumentException(s"invalid chain hash $chainHash") + } + private val rootKey = DeterministicWallet.derivePrivateKey(master, KeyPath(rootPath)) + + override def masterPubKey(account: Long): String = { + val prefix = chainHash match { + case Block.RegtestGenesisBlock.hash | Block.TestnetGenesisBlock.hash | Block.SignetGenesisBlock.hash => vpub + case Block.LivenetGenesisBlock.hash => zpub + case _ => throw new IllegalArgumentException(s"invalid chain hash $chainHash") + } + // master pubkey for account 0 is m/84'/{0'/1'}/0' + val accountPub = DeterministicWallet.publicKey(DeterministicWallet.derivePrivateKey(rootKey, hardened(account))) + DeterministicWallet.encode(accountPub, prefix) + } + + override def derivePublicKey(keyPath: KeyPath): (Crypto.PublicKey, String) = { + val pub = DeterministicWallet.derivePrivateKey(master, keyPath).publicKey + val address = computeBIP84Address(pub, chainHash) + (pub, address) + } + + override def createWallet(rpcClient: BitcoinJsonRPCClient)(implicit ec: ExecutionContext): Future[Boolean] = { + if (walletTimestamp < (TimestampSecond.now() - 2.hours)) { + logger.warn(s"eclair-backed wallet descriptors for wallet=$wallet are too old to be automatically imported into bitcoin core, you will need to manually import them and select how far back to rescan") + Future.successful(false) + } else { + logger.info(s"creating a new on-chain eclair-backed wallet in bitcoind: $wallet") + rpcClient.invoke("createwallet", wallet, /* disable_private_keys */ true, /* blank */ true, /* passphrase */ "", /* avoid_reuse */ false, /* descriptors */ true, /* load_on_startup */ true).flatMap(_ => { + logger.info(s"importing new descriptors ${descriptors(0).descriptors}") + rpcClient.invoke("importdescriptors", descriptors(0).descriptors).collect { + case JArray(results) => results.forall(item => { + val JBool(success) = item \ "success" + success + }) + } + }).recover { e => + logger.error("cannot create eclair-backed on-chain wallet: ", e) + false + } + } + } + + override def descriptors(account: Long): Descriptors = { + // TODO: we should use 'h' everywhere once bitcoin-kmp supports it. + val keyPath = s"$rootPath/$account'".replace('\'', 'h') + val prefix = chainHash match { + case Block.LivenetGenesisBlock.hash => xpub + case _ => tpub + } + val accountPub = DeterministicWallet.publicKey(DeterministicWallet.derivePrivateKey(rootKey, hardened(account))) + // descriptors for account 0 are: + // 84'/{0'/1'}/0'/0/* for main addresses + // 84'/{0'/1'}/0'/1/* for change addresses + val receiveDesc = s"wpkh([$fingerPrintHex/$keyPath]${encode(accountPub, prefix)}/0/*)" + val changeDesc = s"wpkh([$fingerPrintHex/$keyPath]${encode(accountPub, prefix)}/1/*)" + Descriptors(wallet_name = wallet, descriptors = List( + Descriptor(desc = s"$receiveDesc#${descriptorChecksum(receiveDesc)}", internal = false, active = true, timestamp = walletTimestamp.toLong), + Descriptor(desc = s"$changeDesc#${descriptorChecksum(changeDesc)}", internal = true, active = true, timestamp = walletTimestamp.toLong), + )) + } + + override def sign(psbt: Psbt, ourInputs: Seq[Int], ourOutputs: Seq[Int]): Try[Psbt] = { + for { + spent <- spentAmount(psbt, ourInputs) + change <- changeAmount(psbt, ourOutputs) + _ = logger.debug(s"signing txid=${psbt.getGlobal.getTx.txid} fees=${psbt.computeFees()} spent=$spent change=$change") + _ <- Try { + ourOutputs.foreach(i => require(isOurOutput(psbt, i), s"could not verify output $i: bitcoin core may be malicious")) + } + signed <- ourInputs.foldLeft(Try(psbt)) { + case (Failure(psbt), _) => Failure(psbt) + case (Success(psbt), i) => signPsbtInput(psbt, i) + } + } yield signed + } + + private def spentAmount(psbt: Psbt, ourInputs: Seq[Int]): Try[Satoshi] = Try { + ourInputs.map(i => { + val input = psbt.getInput(i) + require(input != null, s"input $i is missing from psbt: bitcoin core may be malicious") + require(input.getWitnessUtxo != null, s"input $i does not have a witness utxo: bitcoin core may be malicious") + fr.acinq.bitcoin.scalacompat.KotlinUtils.kmp2scala(input.getWitnessUtxo.amount) + }).sum + } + + private def changeAmount(psbt: Psbt, ourOutputs: Seq[Int]): Try[Satoshi] = Try { + ourOutputs.map(i => { + require(psbt.getGlobal.getTx.txOut.size() > i, s"output $i is missing from psbt: bitcoin core may be malicious") + fr.acinq.bitcoin.scalacompat.KotlinUtils.kmp2scala(psbt.getGlobal.getTx.txOut.get(i).amount) + }).sum + } + + /** Check that an output belongs to us (i.e. we can recompute its public key from its bip32 path). */ + private def isOurOutput(psbt: Psbt, outputIndex: Int): Boolean = { + import fr.acinq.bitcoin.scalacompat.KotlinUtils._ + if (psbt.getOutputs.size() <= outputIndex || psbt.getGlobal.getTx.txOut.size() <= outputIndex) { + return false + } + val output = psbt.getOutputs.get(outputIndex) + val txOut = psbt.getGlobal.getTx.txOut.get(outputIndex) + output.getDerivationPaths.asScala.headOption match { + case Some((pub, keypath)) => + val (expectedPubKey, _) = derivePublicKey(KeyPath(keypath.getKeyPath.path.asScala.toSeq.map(_.longValue()))) + val expectedScript = Script.write(Script.pay2wpkh(expectedPubKey)) + if (expectedPubKey != kmp2scala(pub)) { + logger.warn(s"public key mismatch (expected=$expectedPubKey, actual=$pub): bitcoin core may be malicious") + false + } else if (kmp2scala(txOut.publicKeyScript) != expectedScript) { + logger.warn(s"script mismatch (expected=$expectedScript, actual=${txOut.publicKeyScript}): bitcoin core may be malicious") + false + } else { + true + } + case None => + logger.warn("derivation path is missing: bitcoin core may be malicious") + false + } + } + + private def signPsbtInput(psbt: Psbt, pos: Int): Try[Psbt] = Try { + import fr.acinq.bitcoin.scalacompat.KotlinUtils._ + import fr.acinq.bitcoin.{Script, SigHash} + + val input = psbt.getInput(pos) + require(input != null, s"input $pos is missing from psbt: bitcoin core may be malicious") + + // For each wallet input, Bitcoin Core will provide: + // - the output that was spent, in the PSBT's witness utxo field + // - the actual transaction that was spent, in the PSBT's non-witness utxo field + // We check that these fields are consistent and match the outpoint that is spent in the PSBT. + // This prevents attacks where Bitcoin Core would lie about the amount being spent and make us pay very high fees. + require(input.getNonWitnessUtxo != null, "non-witness utxo is missing: bitcoin core may be malicious") + require(input.getNonWitnessUtxo.txid == psbt.getGlobal.getTx.txIn.get(pos).outPoint.txid, "utxo txid mismatch: bitcoin core may be malicious") + require(input.getNonWitnessUtxo.txOut.get(psbt.getGlobal.getTx.txIn.get(pos).outPoint.index.toInt) == input.getWitnessUtxo, "utxo mismatch: bitcoin core may be malicious") + + // We must use SIGHASH_ALL, otherwise we would be vulnerable to "signature reuse" attacks. + // When unspecified, the sighash used will be SIGHASH_ALL. + require(Option(input.getSighashType).forall(_ == SigHash.SIGHASH_ALL), s"input sighash must be SIGHASH_ALL (got=${input.getSighashType}): bitcoin core may be malicious") + + // Check that we're signing a p2wpkh input and that the keypath is provided and correct. + require(input.getDerivationPaths.size() == 1, "bip32 derivation path is missing: bitcoin core may be malicious") + val (pub, keypath) = input.getDerivationPaths.asScala.toSeq.head + val priv = fr.acinq.bitcoin.DeterministicWallet.derivePrivateKey(master.priv, keypath.getKeyPath).getPrivateKey + require(priv.publicKey() == pub, s"derived public key doesn't match (expected=$pub actual=${priv.publicKey()}): bitcoin core may be malicious") + val expectedScript = ByteVector(Script.write(Script.pay2wpkh(pub))) + require(kmp2scala(input.getWitnessUtxo.publicKeyScript) == expectedScript, s"script mismatch (expected=$expectedScript, actual=${input.getWitnessUtxo.publicKeyScript}): bitcoin core may be malicious") + + // Update the input with the right script for a p2wpkh input, which is a *p2pkh* script, then sign and finalize. + val updated: Either[UpdateFailure, Psbt] = psbt.updateWitnessInput(psbt.getGlobal.getTx.txIn.get(pos).outPoint, input.getWitnessUtxo, null, Script.pay2pkh(pub), SigHash.SIGHASH_ALL, input.getDerivationPaths) + val signed = updated.flatMap(_.sign(priv, pos)) + val finalized = signed.flatMap(s => { + require(s.getSig.last.toInt == SigHash.SIGHASH_ALL, "signature must end with SIGHASH_ALL") + s.getPsbt.finalizeWitnessInput(pos, Script.witnessPay2wpkh(pub, s.getSig)) + }) + finalized match { + case Right(psbt) => psbt + case Left(failure) => throw new RuntimeException(s"cannot sign psbt input, error = $failure") + } + } +} diff --git a/eclair-core/src/main/scala/fr/acinq/eclair/crypto/keymanager/LocalOnchainKeyManager.scala b/eclair-core/src/main/scala/fr/acinq/eclair/crypto/keymanager/LocalOnchainKeyManager.scala deleted file mode 100644 index d92fb83e38..0000000000 --- a/eclair-core/src/main/scala/fr/acinq/eclair/crypto/keymanager/LocalOnchainKeyManager.scala +++ /dev/null @@ -1,180 +0,0 @@ -package fr.acinq.eclair.crypto.keymanager - -import com.typesafe.config.ConfigFactory -import fr.acinq.bitcoin.ScriptWitness -import fr.acinq.bitcoin.psbt.{Psbt, UpdateFailure} -import fr.acinq.bitcoin.scalacompat.DeterministicWallet._ -import fr.acinq.bitcoin.scalacompat.{Block, ByteVector32, Crypto, DeterministicWallet, MnemonicCode, computeBIP84Address} -import fr.acinq.eclair.TimestampSecond -import fr.acinq.eclair.blockchain.bitcoind.rpc.BitcoinCoreClient.{Descriptor, Descriptors} -import grizzled.slf4j.Logging -import scodec.bits.ByteVector - -import java.io.File -import scala.jdk.CollectionConverters.MapHasAsScala -import scala.util.Try - -object LocalOnchainKeyManager extends Logging { - def descriptorChecksum(span: String): String = fr.acinq.bitcoin.Descriptor.checksum(span) - - /** - * Load a configuration file and create an onchain key manager - * - * @param datadir eclair data directory - * @param chainHash chain we're on - * @return a LocalOnchainKeyManager instance if a configuration file exists - */ - def load(datadir: File, chainHash: ByteVector32): Option[LocalOnchainKeyManager] = { - // we use a specific file instead of adding values to eclair's configuration file because it is available everywhere in the code through - // the actor system's settings and we'd like to restrict access to the onchain wallet seed - val file = new File(datadir, "eclair-signer.conf") - if (file.exists()) { - val config = ConfigFactory.parseFile(file) - val wallet = config.getString("eclair.signer.wallet") - val mnemonics = config.getString("eclair.signer.mnemonics") - val passphrase = config.getString("eclair.signer.passphrase") - val timestamp = config.getLong("eclair.signer.timestamp") - val keyManager = new LocalOnchainKeyManager(wallet, MnemonicCode.toSeed(mnemonics, passphrase), TimestampSecond(timestamp), chainHash) - logger.info(s"using onchain key manager wallet=${wallet} xpub=${keyManager.getOnchainMasterPubKey(0)}") - Some(keyManager) - } else { - None - } - } -} - -class LocalOnchainKeyManager(override val wallet: String, seed: ByteVector, timestamp: TimestampSecond, chainHash: ByteVector32) extends OnchainKeyManager with Logging { - - import LocalOnchainKeyManager._ - - // master key. we will use it to generate a BIP84 wallet that can be used: - // - to generate a watch-only wallet with any BIP84-compatible bitcoin wallet - // - to generate descriptors that can be import into Bitcoin Core to create a watch-only wallet which can be used - // by Eclair to fund transactions (only Eclair will be able to sign wallet inputs). - - // m / purpose' / coin_type' / account' / change / address_index - private val master = DeterministicWallet.generate(seed) - private val fingerprint = DeterministicWallet.fingerprint(master) & 0xFFFFFFFFL - private val fingerPrintHex = String.format("%8s", fingerprint.toHexString).replace(' ', '0') - // root bip32 onchain path - // we use BIP84 (p2wpkh) path: 84'/{0'/1'} - private val rootPath = chainHash match { - case Block.RegtestGenesisBlock.hash | Block.TestnetGenesisBlock.hash | Block.SignetGenesisBlock.hash => "84'/1'" - case Block.LivenetGenesisBlock.hash => "84'/0'" - case _ => throw new IllegalArgumentException(s"invalid chain hash ${chainHash}") - } - private val rootKey = DeterministicWallet.derivePrivateKey(master, KeyPath(rootPath)) - - - override def getOnchainMasterPubKey(account: Long): String = { - val prefix = chainHash match { - case Block.RegtestGenesisBlock.hash | Block.TestnetGenesisBlock.hash | Block.SignetGenesisBlock.hash => vpub - case Block.LivenetGenesisBlock.hash => zpub - case _ => throw new IllegalArgumentException(s"invalid chain hash ${chainHash}") - } - // master pubkey for account 0 is m/84'/{0'/1'}/0' - val accountPub = DeterministicWallet.publicKey(DeterministicWallet.derivePrivateKey(rootKey, hardened(account))) - DeterministicWallet.encode(accountPub, prefix) - } - - override def getWalletTimestamp(): TimestampSecond = timestamp - - override def getDescriptors(account: Long): Descriptors = { - val keyPath = s"$rootPath/$account'".replace('\'', 'h') // Bitcoin Core understands both ' and h suffix for hardened derivation, and h is much easier to parse for external tools - val prefix: Int = chainHash match { - case Block.LivenetGenesisBlock.hash => xpub - case _ => tpub - } - val accountPub = DeterministicWallet.publicKey(DeterministicWallet.derivePrivateKey(rootKey, hardened(account))) - // descriptors for account 0 are: - // 84'/{0'/1'}/0'/0/* for main addresses - // 84'/{0'/1'}/0'/1/* for change addresses - val receiveDesc = s"wpkh([${this.fingerPrintHex}/$keyPath]${encode(accountPub, prefix)}/0/*)" - val changeDesc = s"wpkh([${this.fingerPrintHex}/$keyPath]${encode(accountPub, prefix)}/1/*)" - - Descriptors(wallet_name = wallet, descriptors = List( - Descriptor(desc = s"$receiveDesc#${descriptorChecksum(receiveDesc)}", internal = false, active = true, timestamp = timestamp.toLong), - Descriptor(desc = s"$changeDesc#${descriptorChecksum(changeDesc)}", internal = true, active = true, timestamp = timestamp.toLong), - )) - } - - override def signPsbt(psbt: Psbt, ourInputs: Seq[Int], ourOutputs: Seq[Int]): Try[Psbt] = Try { - import fr.acinq.bitcoin.scalacompat.KotlinUtils._ - - val spent = ourInputs.map(i => kmp2scala(psbt.getInput(i).getWitnessUtxo.amount)).sum - val backtous = ourOutputs.map(i => kmp2scala(psbt.getGlobal.getTx.txOut.get(i).amount)).sum - - logger.info(s"signing ${psbt.getGlobal.getTx.txid} fees ${psbt.computeFees()} spent $spent to_us $backtous") - ourOutputs.foreach(i => isOurOutput(psbt, i)) - ourInputs.foldLeft(psbt) { (p, i) => sigbnPsbtInput(p, i) } - } - - - override def getPublicKey(keyPath: KeyPath): (Crypto.PublicKey, String) = { - import fr.acinq.bitcoin.scalacompat.KotlinUtils._ - val pub = getPrivateKey(keyPath.keyPath).publicKey() - val address = computeBIP84Address(pub, chainHash) - (pub, address) - } - - private def getPrivateKey(keyPath: fr.acinq.bitcoin.KeyPath) = fr.acinq.bitcoin.DeterministicWallet.derivePrivateKey(master.priv, keyPath).getPrivateKey - - // check that an output belongs to us i.e. we can recompute its public from its bip32 path - private def isOurOutput(psbt: Psbt, outputIndex: Int) = { - val output = psbt.getOutputs.get(outputIndex) - val txout = psbt.getGlobal.getTx.txOut.get(outputIndex) - output.getDerivationPaths.size() match { - case 1 => - output.getDerivationPaths.asScala.foreach { case (pub, keypath) => - val check = getPrivateKey(keypath.getKeyPath).publicKey() - require(pub == check, s"cannot compute public key for $txout") - require(txout.publicKeyScript.contentEquals(fr.acinq.bitcoin.Script.write(fr.acinq.bitcoin.Script.pay2wpkh(pub))), s"output pubkeyscript does not match ours for $txout") - } - case _ => throw new IllegalArgumentException(s"cannot verify that $txout sends to us") - } - } - - private def sigbnPsbtInput(psbt: Psbt, pos: Int): Psbt = { - import fr.acinq.bitcoin.scalacompat.KotlinUtils.eitherkmp2either - import fr.acinq.bitcoin.{Script, SigHash} - - val input = psbt.getInput(pos) - - // For each wallet input, Bitcoin Core will provide - // - the output that was spent, in the PSBT's witness utxo field - // - the actual transaction that was spent, in the PSBT's non-witness utxo field - // we check that this fields are consistent and match the outpoint that is spent in the PSBT - // this prevents attacks where bitcoin core would lie about the amount being spent and make us pay very high fees - require(input.getNonWitnessUtxo != null, "non-witness utxo is missing") - require(input.getNonWitnessUtxo.txid == psbt.getGlobal.getTx.txIn.get(pos).outPoint.txid, "utxo txid mismatch") - require(input.getNonWitnessUtxo.txOut.get(psbt.getGlobal.getTx.txIn.get(pos).outPoint.index.toInt) == input.getWitnessUtxo, "utxo mismatch") - - // not using SIGHASH_ALL would make us vulnerable to "signature reuse" attacks - // here null means unspecified means SIGHASH_ALL - require(Option(input.getSighashType).forall(_ == SigHash.SIGHASH_ALL), "input sighashtype must be SIGHASH_ALL") - - // check that we're signing a p2wpkh input and that the keypath is provided and correct - require(Script.isPay2wpkh(input.getWitnessUtxo.publicKeyScript.toByteArray), "spent input is not p2wpkh") - require(input.getDerivationPaths.size() == 1, "invalid bip32 path") - val (pub, keypath) = input.getDerivationPaths.asScala.toSeq.head - - // use provided bip32 path to compute the private key - val priv = fr.acinq.bitcoin.DeterministicWallet.derivePrivateKey(master.priv, keypath.getKeyPath).getPrivateKey - require(priv.publicKey() == pub, "cannot compute private key") - - // update the input with the right script for a p2wpkh input, which is a * p2pkh * script - // then sign and finalize the psbt input - - val updated: Either[UpdateFailure, Psbt] = psbt.updateWitnessInput(psbt.getGlobal.getTx.txIn.get(pos).outPoint, input.getWitnessUtxo, null, Script.pay2pkh(pub), SigHash.SIGHASH_ALL, input.getDerivationPaths) - val signed = updated.flatMap(_.sign(priv, pos)) - val finalized = signed.flatMap(s => { - val sig = s.getSig - require(sig.get(sig.size() - 1).toInt == SigHash.SIGHASH_ALL, "signature must end with SIGHASH_ALL") - s.getPsbt.finalizeWitnessInput(pos, new ScriptWitness().push(sig).push(pub.value)) - }) - finalized match { - case Right(psbt) => psbt - case Left(failure) => throw new RuntimeException(s"cannot sign psbt input, error = $failure") - } - } -} diff --git a/eclair-core/src/main/scala/fr/acinq/eclair/crypto/keymanager/OnChainKeyManager.scala b/eclair-core/src/main/scala/fr/acinq/eclair/crypto/keymanager/OnChainKeyManager.scala new file mode 100644 index 0000000000..867289b4bd --- /dev/null +++ b/eclair-core/src/main/scala/fr/acinq/eclair/crypto/keymanager/OnChainKeyManager.scala @@ -0,0 +1,70 @@ +/* + * Copyright 2023 ACINQ SAS + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package fr.acinq.eclair.crypto.keymanager + +import fr.acinq.bitcoin.psbt.Psbt +import fr.acinq.bitcoin.scalacompat.Crypto.PublicKey +import fr.acinq.bitcoin.scalacompat.DeterministicWallet.KeyPath +import fr.acinq.eclair.TimestampSecond +import fr.acinq.eclair.blockchain.bitcoind.rpc.BitcoinCoreClient.Descriptors +import fr.acinq.eclair.blockchain.bitcoind.rpc.BitcoinJsonRPCClient + +import scala.concurrent.{ExecutionContext, Future} +import scala.util.Try + +trait OnChainKeyManager { + def wallet: String + + /** + * @return the creation time of the wallet managed by this key manager + */ + def walletTimestamp(): TimestampSecond + + /** + * Create a bitcoin core watch-only wallet with private keys owned by this key manager instance. + * This should only be called if the corresponding wallet doesn't already exist. + */ + def createWallet(rpcClient: BitcoinJsonRPCClient)(implicit ec: ExecutionContext): Future[Boolean] + + /** + * @param account account number (0 is used by most wallets) + * @return the on-chain pubkey for this account, which can then be imported into a BIP39-compatible wallet such as Electrum + */ + def masterPubKey(account: Long): String + + /** + * @param keyPath BIP32 path + * @return the (public key, address) pair for this BIP32 path starting from the master key + */ + def derivePublicKey(keyPath: KeyPath): (PublicKey, String) + + /** + * @param account account number + * @return a pair of (main, change) wallet descriptors that can be imported into an on-chain wallet + */ + def descriptors(account: Long): Descriptors + + /** + * Sign the inputs provided in [[ourInputs]] and verifies that [[ourOutputs]] belong to our bitcoin wallet. + * + * @param psbt input psbt + * @param ourInputs index of inputs that belong to our on-chain wallet and need to be signed + * @param ourOutputs index of outputs that belong to our on-chain wallet + * @return a signed psbt, where all our inputs are signed + */ + def sign(psbt: Psbt, ourInputs: Seq[Int], ourOutputs: Seq[Int]): Try[Psbt] +} diff --git a/eclair-core/src/main/scala/fr/acinq/eclair/crypto/keymanager/OnchainKeyManager.scala b/eclair-core/src/main/scala/fr/acinq/eclair/crypto/keymanager/OnchainKeyManager.scala deleted file mode 100644 index 07008e70bf..0000000000 --- a/eclair-core/src/main/scala/fr/acinq/eclair/crypto/keymanager/OnchainKeyManager.scala +++ /dev/null @@ -1,49 +0,0 @@ -package fr.acinq.eclair.crypto.keymanager - -import fr.acinq.bitcoin.psbt.Psbt -import fr.acinq.bitcoin.scalacompat.Crypto.PublicKey -import fr.acinq.bitcoin.scalacompat.DeterministicWallet.KeyPath -import fr.acinq.eclair.TimestampSecond -import fr.acinq.eclair.blockchain.bitcoind.rpc.BitcoinCoreClient.Descriptors - -import scala.util.Try - -trait OnchainKeyManager { - def wallet: String - - /** - * - * @param account account number (0 is used by most wallets) - * @return the onchain pubkey for this account, which can then be imported into a BIP39-compatible wallet such as Electrum - */ - def getOnchainMasterPubKey(account: Long): String - - /** - * - * @param keyPath BIP path - * @return the (public key, address) pair for this BIP32 path - */ - def getPublicKey(keyPath: KeyPath): (PublicKey, String) - - /** - * - * @return the creation time of the wallet managed by this key manager - */ - def getWalletTimestamp(): TimestampSecond - - /** - * - * @param account account number - * @return a pair of (main, change) wallet descriptors that can be imported into an onchain wallet - */ - def getDescriptors(account: Long): Descriptors - - /** - * - * @param psbt input psbt - * @param ourInputs index of inputs that belong to our onchain wallet and need to be signed - * @param ourOutputs index of outputs that belong to our onchain wallet - * @return a signed psbt, where all our inputs are signed - */ - def signPsbt(psbt: Psbt, ourInputs: Seq[Int], ourOutputs: Seq[Int]): Try[Psbt] -} diff --git a/eclair-core/src/test/scala/fr/acinq/eclair/StartupSpec.scala b/eclair-core/src/test/scala/fr/acinq/eclair/StartupSpec.scala index ae8b9cbf32..5f20a00a69 100644 --- a/eclair-core/src/test/scala/fr/acinq/eclair/StartupSpec.scala +++ b/eclair-core/src/test/scala/fr/acinq/eclair/StartupSpec.scala @@ -42,7 +42,7 @@ class StartupSpec extends AnyFunSuite { val nodeKeyManager = new LocalNodeKeyManager(randomBytes32(), chainHash = Block.TestnetGenesisBlock.hash) val channelKeyManager = new LocalChannelKeyManager(randomBytes32(), chainHash = Block.TestnetGenesisBlock.hash) val db = TestDatabases.inMemoryDb() - NodeParams.makeNodeParams(conf, UUID.fromString("01234567-0123-4567-89ab-0123456789ab"), nodeKeyManager, channelKeyManager, None, db, blockCount, feerates) + NodeParams.makeNodeParams(conf, UUID.fromString("01234567-0123-4567-89ab-0123456789ab"), nodeKeyManager, channelKeyManager, None, None, db, blockCount, feerates) } test("check configuration") { diff --git a/eclair-core/src/test/scala/fr/acinq/eclair/TestConstants.scala b/eclair-core/src/test/scala/fr/acinq/eclair/TestConstants.scala index 21bd98694c..e84425cd4e 100644 --- a/eclair-core/src/test/scala/fr/acinq/eclair/TestConstants.scala +++ b/eclair-core/src/test/scala/fr/acinq/eclair/TestConstants.scala @@ -81,6 +81,7 @@ object TestConstants { def nodeParams: NodeParams = NodeParams( nodeKeyManager, channelKeyManager, + onChainKeyManager_opt = None, blockHeight = new AtomicLong(defaultBlockHeight), feerates = new AtomicReference(FeeratesPerKw.single(feeratePerKw)), alias = "alice", @@ -244,6 +245,7 @@ object TestConstants { def nodeParams: NodeParams = NodeParams( nodeKeyManager, channelKeyManager, + onChainKeyManager_opt = None, blockHeight = new AtomicLong(defaultBlockHeight), feerates = new AtomicReference(FeeratesPerKw.single(feeratePerKw)), alias = "bob", diff --git a/eclair-core/src/test/scala/fr/acinq/eclair/blockchain/DummyOnChainWallet.scala b/eclair-core/src/test/scala/fr/acinq/eclair/blockchain/DummyOnChainWallet.scala index 8fe0f72281..8caca5b45b 100644 --- a/eclair-core/src/test/scala/fr/acinq/eclair/blockchain/DummyOnChainWallet.scala +++ b/eclair-core/src/test/scala/fr/acinq/eclair/blockchain/DummyOnChainWallet.scala @@ -21,10 +21,8 @@ import fr.acinq.bitcoin.psbt.Psbt import fr.acinq.bitcoin.scalacompat.Crypto.PublicKey import fr.acinq.bitcoin.scalacompat.{ByteVector32, Crypto, OutPoint, Satoshi, SatoshiLong, Script, Transaction, TxIn, TxOut} import fr.acinq.bitcoin.{Bech32, SigHash, SigVersion} -import fr.acinq.eclair.blockchain.OnChainWallet.{FundTransactionResponse, MakeFundingTxResponse, OnChainBalance} +import fr.acinq.eclair.blockchain.OnChainWallet.{FundTransactionResponse, MakeFundingTxResponse, OnChainBalance, ProcessPsbtResponse} import fr.acinq.eclair.blockchain.bitcoind.BitcoindService.SignTransactionResponse -import fr.acinq.eclair.blockchain.bitcoind.rpc.BitcoinCoreClient -import fr.acinq.eclair.blockchain.bitcoind.rpc.BitcoinCoreClient.ProcessPsbtResponse import fr.acinq.eclair.blockchain.fee.FeeratePerKw import fr.acinq.eclair.transactions.Transactions import fr.acinq.eclair.{randomBytes32, randomKey} @@ -55,7 +53,7 @@ class DummyOnChainWallet extends OnChainWallet with OnchainPubkeyCache { Future.successful(FundTransactionResponse(tx, 0 sat, None)) } - override def signPsbt(psbt: Psbt, ourInputs: Seq[Int], ourOutputs: Seq[Int])(implicit ec: ExecutionContext): Future[BitcoinCoreClient.ProcessPsbtResponse] = Future.successful(ProcessPsbtResponse(psbt, complete = true)) + override def signPsbt(psbt: Psbt, ourInputs: Seq[Int], ourOutputs: Seq[Int])(implicit ec: ExecutionContext): Future[ProcessPsbtResponse] = Future.successful(ProcessPsbtResponse(psbt, complete = true)) override def publishTransaction(tx: Transaction)(implicit ec: ExecutionContext): Future[ByteVector32] = { published += (tx.txid -> tx) @@ -155,7 +153,7 @@ class SingleKeyOnChainWallet extends OnChainWallet with OnchainPubkeyCache { Future.successful(FundTransactionResponse(fundedTx, fee, Some(tx.txOut.length))) } - private def signTransaction(tx: Transaction, allowIncomplete: Boolean)(implicit ec: ExecutionContext): Future[SignTransactionResponse] = { + private def signTransaction(tx: Transaction): Future[SignTransactionResponse] = { val signedTx = tx.txIn.zipWithIndex.foldLeft(tx) { case (currentTx, (txIn, index)) => inputs.find(_.txid == txIn.outPoint.txid) match { case Some(inputTx) => @@ -181,9 +179,8 @@ class SingleKeyOnChainWallet extends OnChainWallet with OnchainPubkeyCache { case (currentPsbt, (txIn, index)) => inputs.find(_.txid == txIn.outPoint.txid) match { case Some(inputTx) => val sig = Transaction.signInput(tx, index, Script.pay2pkh(pubkey), SigHash.SIGHASH_ALL, inputTx.txOut.head.amount, SigVersion.SIGVERSION_WITNESS_V0, privkey) - val updated = currentPsbt.updateWitnessInput(txIn.outPoint, inputTx.txOut(txIn.outPoint.index.toInt), null, Script.pay2pkh(pubkey).map(scala2kmp).asJava, null, java.util.Map.of()) - val finalized = updated.getRight.finalizeWitnessInput(txIn.outPoint, Script.witnessPay2wpkh(pubkey, sig)) - finalized.getRight + val updated = currentPsbt.updateWitnessInput(txIn.outPoint, inputTx.txOut(txIn.outPoint.index.toInt), null, Script.pay2pkh(pubkey).map(scala2kmp).asJava, null, java.util.Map.of()).getRight + updated.finalizeWitnessInput(txIn.outPoint, Script.witnessPay2wpkh(pubkey, sig)).getRight case None => currentPsbt } } @@ -195,7 +192,7 @@ class SingleKeyOnChainWallet extends OnChainWallet with OnchainPubkeyCache { val tx = Transaction(2, Nil, Seq(TxOut(amount, pubkeyScript)), 0) for { fundedTx <- fundTransaction(tx, feeRatePerKw, replaceable = true) - signedTx <- signTransaction(fundedTx.tx, allowIncomplete = true) + signedTx <- signTransaction(fundedTx.tx) } yield MakeFundingTxResponse(signedTx.tx, 0, fundedTx.fee) } diff --git a/eclair-core/src/test/scala/fr/acinq/eclair/blockchain/bitcoind/BitcoinCoreClientSpec.scala b/eclair-core/src/test/scala/fr/acinq/eclair/blockchain/bitcoind/BitcoinCoreClientSpec.scala index 073cb06fdb..505340a82b 100644 --- a/eclair-core/src/test/scala/fr/acinq/eclair/blockchain/bitcoind/BitcoinCoreClientSpec.scala +++ b/eclair-core/src/test/scala/fr/acinq/eclair/blockchain/bitcoind/BitcoinCoreClientSpec.scala @@ -24,14 +24,14 @@ import fr.acinq.bitcoin.psbt.{Psbt, UpdateFailure} import fr.acinq.bitcoin.scalacompat.Crypto.PublicKey import fr.acinq.bitcoin.scalacompat.{Block, Btc, BtcDouble, ByteVector32, Crypto, DeterministicWallet, MilliBtcDouble, MnemonicCode, OP_DROP, OP_PUSHDATA, OutPoint, Satoshi, SatoshiLong, Script, ScriptWitness, Transaction, TxIn, TxOut, addressFromPublicKeyScript, addressToPublicKeyScript, computeBIP84Address, computeP2PkhAddress, computeP2WpkhAddress} import fr.acinq.bitcoin.{Bech32, SigHash, SigVersion} -import fr.acinq.eclair.blockchain.OnChainWallet.{FundTransactionResponse, MakeFundingTxResponse, OnChainBalance} +import fr.acinq.eclair.blockchain.OnChainWallet.{FundTransactionResponse, MakeFundingTxResponse, OnChainBalance, ProcessPsbtResponse} import fr.acinq.eclair.blockchain.WatcherSpec.{createSpendManyP2WPKH, createSpendP2WPKH} import fr.acinq.eclair.blockchain.bitcoind.BitcoindService.{BitcoinReq, SignTransactionResponse} import fr.acinq.eclair.blockchain.bitcoind.rpc.BitcoinCoreClient._ import fr.acinq.eclair.blockchain.bitcoind.rpc.BitcoinJsonRPCAuthMethod.UserPassword import fr.acinq.eclair.blockchain.bitcoind.rpc.{BasicBitcoinJsonRPCClient, BitcoinCoreClient, BitcoinJsonRPCClient, JsonRPCError} import fr.acinq.eclair.blockchain.fee.{FeeratePerByte, FeeratePerKw} -import fr.acinq.eclair.crypto.keymanager.LocalOnchainKeyManager +import fr.acinq.eclair.crypto.keymanager.LocalOnChainKeyManager import fr.acinq.eclair.transactions.{Scripts, Transactions} import fr.acinq.eclair.{BlockHeight, TestConstants, TestKitBaseClass, TimestampSecond, randomBytes32, randomKey} import grizzled.slf4j.Logging @@ -63,17 +63,14 @@ class BitcoinCoreClientSpec extends TestKitBaseClass with BitcoindService with A assume(!useEclairSigner) val sender = TestProbe() - val bitcoinClient = makeBitcoinCoreClient + val bitcoinClient = makeBitcoinCoreClient() val walletPassword = Random.alphanumeric.take(8).mkString sender.send(bitcoincli, BitcoinReq("encryptwallet", walletPassword)) sender.expectMsgType[JString](60 seconds) restartBitcoind(sender) - val f = for { - address <- bitcoinClient.getReceiveAddress() - txid <- bitcoinClient.sendToPubkeyScript(addressToPublicKeyScript(Block.RegtestGenesisBlock.hash, address).toOption.get, 10000.sat, FeeratePerKw(FeeratePerByte(3.sat))) - } yield txid - f.pipeTo(sender.ref) + val pubkeyScript = Script.write(Script.pay2wsh(Scripts.multiSig2of2(randomKey().publicKey, randomKey().publicKey))) + bitcoinClient.makeFundingTx(pubkeyScript, 50 millibtc, FeeratePerKw(10000 sat)).pipeTo(sender.ref) val error = sender.expectMsgType[Failure].cause.asInstanceOf[JsonRPCError].error assert(error.message.contains("Please enter the wallet passphrase with walletpassphrase first")) @@ -84,7 +81,7 @@ class BitcoinCoreClientSpec extends TestKitBaseClass with BitcoindService with A test("fund transactions") { val sender = TestProbe() - val bitcoinClient = makeBitcoinCoreClient + val bitcoinClient = makeBitcoinCoreClient() val txToRemote = { val txNotFunded = Transaction(2, Nil, TxOut(150000 sat, Script.pay2wpkh(randomKey().publicKey)) :: Nil, 0) @@ -150,10 +147,10 @@ class BitcoinCoreClientSpec extends TestKitBaseClass with BitcoindService with A sender.expectMsg(true) } { - // check that bitcoin core is not lying to us + // we check that bitcoin core is not malicious and trying to steal funds. val txNotFunded = Transaction(2, Nil, TxOut(150000 sat, Script.pay2wpkh(randomKey().publicKey)) :: Nil, 0) - def makeEvilBitcoinClient(changePosMod: (Int) => Int, txMod: Transaction => Transaction): BitcoinCoreClient = { + def makeEvilBitcoinClient(changePosMod: Int => Int, txMod: Transaction => Transaction): BitcoinCoreClient = { val badRpcClient = new BitcoinJsonRPCClient { override def wallet: Option[String] = if (useEclairSigner) Some("eclair") else None @@ -166,36 +163,27 @@ class BitcoinCoreClientSpec extends TestKitBaseClass with BitcoindService with A case _ => bitcoinClient.rpcClient.invoke(method, params: _*)(ec) } } - new BitcoinCoreClient(badRpcClient, if (useEclairSigner) Some(onchainKeyManager) else None) + new BitcoinCoreClient(badRpcClient, if (useEclairSigner) Some(onChainKeyManager) else None) } - // verify that bitcoin core is not lying to us - { - val bitcoinClient1 = makeEvilBitcoinClient((pos: Int) => -1, (tx: Transaction) => tx) - bitcoinClient1.fundTransaction(txNotFunded, FundTransactionOptions(TestConstants.feeratePerKw)).pipeTo(sender.ref) - sender.expectMsgType[Failure] - } { - val bitcoinClient1 = makeEvilBitcoinClient((pos: Int) => pos, (tx: Transaction) => tx.copy(txOut = tx.txOut.reverse)) - bitcoinClient1.fundTransaction(txNotFunded, FundTransactionOptions(TestConstants.feeratePerKw)).pipeTo(sender.ref) + // bitcoin core doesn't specify change position. + val evilBitcoinClient = makeEvilBitcoinClient(_ => -1, tx => tx) + evilBitcoinClient.fundTransaction(txNotFunded, FundTransactionOptions(TestConstants.feeratePerKw)).pipeTo(sender.ref) sender.expectMsgType[Failure] } { - val bitcoinClient1 = makeEvilBitcoinClient((pos: Int) => pos, (tx: Transaction) => tx.copy(txOut = tx.txOut.head +: tx.txOut)) - bitcoinClient1.fundTransaction(txNotFunded, FundTransactionOptions(TestConstants.feeratePerKw)).pipeTo(sender.ref) + // bitcoin core tries to send twice the amount we wanted by duplicating the output. + val evilBitcoinClient = makeEvilBitcoinClient(pos => pos, tx => tx.copy(txOut = tx.txOut ++ txNotFunded.txOut)) + evilBitcoinClient.fundTransaction(txNotFunded, FundTransactionOptions(TestConstants.feeratePerKw)).pipeTo(sender.ref) sender.expectMsgType[Failure] } { - val bitcoinClient1 = makeEvilBitcoinClient((pos: Int) => 1, (tx: Transaction) => tx.copy(txOut = tx.txOut.reverse)) - bitcoinClient1.fundTransaction(txNotFunded, FundTransactionOptions(TestConstants.feeratePerKw, changePosition = Some(0))).pipeTo(sender.ref) + // bitcoin core ignores our specified change position. + val evilBitcoinClient = makeEvilBitcoinClient(_ => 1, tx => tx.copy(txOut = tx.txOut.reverse)) + evilBitcoinClient.fundTransaction(txNotFunded, FundTransactionOptions(TestConstants.feeratePerKw, changePosition = Some(0))).pipeTo(sender.ref) sender.expectMsgType[Failure] } - { - val bitcoinClient1 = makeEvilBitcoinClient((pos: Int) => pos, (tx: Transaction) => tx) - bitcoinClient1.fundTransaction(txNotFunded, FundTransactionOptions(TestConstants.feeratePerKw, changePosition = Some(0))).pipeTo(sender.ref) - bitcoinClient.rollback(sender.expectMsgType[FundTransactionResponse].tx).pipeTo(sender.ref) - sender.expectMsg(true) - } } } @@ -203,7 +191,7 @@ class BitcoinCoreClientSpec extends TestKitBaseClass with BitcoindService with A import fr.acinq.bitcoin.scalacompat.KotlinUtils._ val sender = TestProbe() - val defaultWallet = makeBitcoinCoreClient + val defaultWallet = makeBitcoinCoreClient() val walletExternalFunds = new BitcoinCoreClient(createWallet("external_inputs", sender)) // We receive some funds on an address that belongs to our wallet. Seq(25 millibtc, 15 millibtc, 20 millibtc).foreach(amount => { @@ -222,7 +210,7 @@ class BitcoinCoreClientSpec extends TestKitBaseClass with BitcoindService with A defaultWallet.fundTransaction(txNotFunded, FundTransactionOptions(FeeratePerKw(2500 sat), changePosition = Some(1))).pipeTo(sender.ref) val fundedTx = sender.expectMsgType[FundTransactionResponse].tx defaultWallet.signPsbt(new Psbt(fundedTx), fundedTx.txIn.indices, Nil).pipeTo(sender.ref) - val signedTx = sender.expectMsgType[ProcessPsbtResponse].finalTx + val signedTx = sender.expectMsgType[ProcessPsbtResponse].finalTx_opt.toOption.get defaultWallet.publishTransaction(signedTx).pipeTo(sender.ref) sender.expectMsg(signedTx.txid) (OutPoint(signedTx, 0), script, signedTx.txOut(0)) @@ -261,9 +249,9 @@ class BitcoinCoreClientSpec extends TestKitBaseClass with BitcoindService with A // And let bitcoind sign the wallet input. walletExternalFunds.signPsbt(new Psbt(fundedTx2.tx), fundedTx2.tx.txIn.indices, Nil).pipeTo(sender.ref) val psbt = sender.expectMsgType[ProcessPsbtResponse].psbt - val updated: Either[UpdateFailure, Psbt] = psbt.updateWitnessInput(outpoint1, txOut1, null, null, null, java.util.Map.of()) - val finalized = updated.flatMap(_.finalizeWitnessInput(0, Script.witnessMultiSigMofN(Seq(alicePriv, bobPriv).map(_.publicKey), Seq(externalSig)))) - val signedTx: Transaction = finalized.flatMap(_.extract()).getOrElse(throw new RuntimeException("cannot compute transaction")) + val signedTx: Transaction = psbt.updateWitnessInput(outpoint1, txOut1, null, null, null, java.util.Map.of()).getRight + .finalizeWitnessInput(0, Script.witnessMultiSigMofN(Seq(alicePriv, bobPriv).map(_.publicKey), Seq(externalSig))).getRight + .extract().getRight walletExternalFunds.publishTransaction(signedTx).pipeTo(sender.ref) sender.expectMsg(signedTx.txid) @@ -292,10 +280,9 @@ class BitcoinCoreClientSpec extends TestKitBaseClass with BitcoindService with A // We sign our external input. val externalSig = Transaction.signInput(fundedTx.tx, 0, inputScript2, SigHash.SIGHASH_ALL, 300_000 sat, SigVersion.SIGVERSION_WITNESS_V0, alicePriv) - - val updated: Either[UpdateFailure, Psbt] = psbt.updateWitnessInput(externalOutpoint, tx2.txOut(0), null, null, null, java.util.Map.of()) - val finalized = updated.flatMap(_.finalizeWitnessInput(0, Script.witnessMultiSigMofN(Seq(alicePriv, carolPriv).map(_.publicKey), Seq(externalSig)))) - val signedTx: Transaction = finalized.flatMap(_.extract()).getOrElse(throw new RuntimeException("cannot compute transaction")) + val signedTx: Transaction = psbt.updateWitnessInput(externalOutpoint, tx2.txOut(0), null, null, null, java.util.Map.of()).getRight + .finalizeWitnessInput(0, Script.witnessMultiSigMofN(Seq(alicePriv, carolPriv).map(_.publicKey), Seq(externalSig))).getRight + .extract().getRight walletExternalFunds.publishTransaction(signedTx).pipeTo(sender.ref) sender.expectMsg(signedTx.txid) @@ -331,9 +318,9 @@ class BitcoinCoreClientSpec extends TestKitBaseClass with BitcoindService with A // We sign our external input. val externalSig = Transaction.signInput(fundedTx.tx, 0, inputScript2, SigHash.SIGHASH_ALL, 300_000 sat, SigVersion.SIGVERSION_WITNESS_V0, alicePriv) - val updated: Either[UpdateFailure, Psbt] = psbt.updateWitnessInput(OutPoint(tx2, 0), tx2.txOut(0), null, null, null, java.util.Map.of()) - val finalized = updated.flatMap(_.finalizeWitnessInput(0, Script.witnessMultiSigMofN(Seq(alicePriv, carolPriv).map(_.publicKey), Seq(externalSig)))) - val signedTx: Transaction = finalized.flatMap(_.extract()).getOrElse(throw new RuntimeException("cannot compute transaction")) + val signedTx: Transaction = psbt.updateWitnessInput(OutPoint(tx2, 0), tx2.txOut(0), null, null, null, java.util.Map.of()).getRight + .finalizeWitnessInput(0, Script.witnessMultiSigMofN(Seq(alicePriv, carolPriv).map(_.publicKey), Seq(externalSig))).getRight + .extract().getRight walletExternalFunds.publishTransaction(signedTx).pipeTo(sender.ref) sender.expectMsg(signedTx.txid) // We have replaced the previous transaction. @@ -380,7 +367,7 @@ class BitcoinCoreClientSpec extends TestKitBaseClass with BitcoindService with A test("create/commit/rollback funding txs") { val sender = TestProbe() - val bitcoinClient = makeBitcoinCoreClient + val bitcoinClient = makeBitcoinCoreClient() bitcoinClient.onChainBalance().pipeTo(sender.ref) assert(sender.expectMsgType[OnChainBalance].confirmed > 0.sat) @@ -431,7 +418,7 @@ class BitcoinCoreClientSpec extends TestKitBaseClass with BitcoindService with A test("ensure feerate is always above min-relay-fee") { val sender = TestProbe() - val bitcoinClient = makeBitcoinCoreClient + val bitcoinClient = makeBitcoinCoreClient() val pubkeyScript = Script.write(Script.pay2wsh(Scripts.multiSig2of2(randomKey().publicKey, randomKey().publicKey))) // 200 sat/kw is below the min-relay-fee @@ -444,7 +431,7 @@ class BitcoinCoreClientSpec extends TestKitBaseClass with BitcoindService with A test("unlock failed funding txs") { val sender = TestProbe() - val bitcoinClient = makeBitcoinCoreClient + val bitcoinClient = makeBitcoinCoreClient() bitcoinClient.onChainBalance().pipeTo(sender.ref) assert(sender.expectMsgType[OnChainBalance].confirmed > 0.sat) @@ -471,7 +458,7 @@ class BitcoinCoreClientSpec extends TestKitBaseClass with BitcoindService with A import fr.acinq.bitcoin.scalacompat.KotlinUtils._ val sender = TestProbe() - val bitcoinClient = makeBitcoinCoreClient + val bitcoinClient = makeBitcoinCoreClient() generateBlocks(1) // generate a block to ensure we start with an empty mempool // create a first transaction with multiple inputs @@ -489,7 +476,7 @@ class BitcoinCoreClientSpec extends TestKitBaseClass with BitcoindService with A sender.expectMsg(fundedTx.txIn.map(_.outPoint).toSet) bitcoinClient.signPsbt(new Psbt(fundedTx), fundedTx.txIn.indices, Nil).pipeTo(sender.ref) - val signedTx = sender.expectMsgType[ProcessPsbtResponse].finalTx + val signedTx = sender.expectMsgType[ProcessPsbtResponse].finalTx_opt.toOption.get bitcoinClient.publishTransaction(signedTx).pipeTo(sender.ref) sender.expectMsg(signedTx.txid) // once the tx is published, the inputs should be automatically unlocked @@ -510,7 +497,7 @@ class BitcoinCoreClientSpec extends TestKitBaseClass with BitcoindService with A sender.expectMsg(fundedTx.txIn.map(_.outPoint).toSet) bitcoinClient.signPsbt(new Psbt(fundedTx), fundedTx.txIn.indices, Nil).pipeTo(sender.ref) - val signedTx = sender.expectMsgType[ProcessPsbtResponse].finalTx + val signedTx = sender.expectMsgType[ProcessPsbtResponse].finalTx_opt.toOption.get bitcoinClient.publishTransaction(signedTx).pipeTo(sender.ref) sender.expectMsg(signedTx.txid) // once the tx is published, the inputs should be automatically unlocked @@ -532,7 +519,7 @@ class BitcoinCoreClientSpec extends TestKitBaseClass with BitcoindService with A val sender = TestProbe() val pubkeyScript = Script.write(Script.pay2wsh(Scripts.multiSig2of2(randomKey().publicKey, randomKey().publicKey))) - val bitcoinClient = makeBitcoinCoreClient + val bitcoinClient = makeBitcoinCoreClient() // create a huge tx so we make sure it has > 2 inputs bitcoinClient.makeFundingTx(pubkeyScript, 250 btc, FeeratePerKw(1000 sat)).pipeTo(sender.ref) @@ -540,16 +527,12 @@ class BitcoinCoreClientSpec extends TestKitBaseClass with BitcoindService with A assert(fundingTx.txIn.length > 2) // spend the first 2 inputs - val tx1 = { - val tx = fundingTx.copy( - txIn = fundingTx.txIn.take(2), - txOut = fundingTx.txOut.updated(outputIndex, fundingTx.txOut(outputIndex).copy(amount = 50 btc)) - ) - bitcoinClient.fundTransaction(tx, FeeratePerKw(1500 sat), true).pipeTo(sender.ref) - sender.expectMsgType[FundTransactionResponse].tx - } + val tx1 = fundingTx.copy( + txIn = fundingTx.txIn.take(2), + txOut = fundingTx.txOut.updated(outputIndex, fundingTx.txOut(outputIndex).copy(amount = 50 btc)) + ) bitcoinClient.signPsbt(new Psbt(tx1), tx1.txIn.indices, Nil).pipeTo(sender.ref) - val tx2 = sender.expectMsgType[ProcessPsbtResponse].finalTx + val tx2 = sender.expectMsgType[ProcessPsbtResponse].finalTx_opt.toOption.get bitcoinClient.commit(tx2).pipeTo(sender.ref) sender.expectMsg(true) @@ -576,14 +559,14 @@ class BitcoinCoreClientSpec extends TestKitBaseClass with BitcoindService with A import fr.acinq.bitcoin.scalacompat.KotlinUtils._ val sender = TestProbe() - val bitcoinClient = makeBitcoinCoreClient + val bitcoinClient = makeBitcoinCoreClient() val txNotFunded = Transaction(2, Nil, Seq(TxOut(200_000 sat, Script.pay2wpkh(randomKey().publicKey))), 0) bitcoinClient.fundTransaction(txNotFunded, FeeratePerKw(FeeratePerByte(1 sat)), replaceable = true).pipeTo(sender.ref) val txFunded1 = sender.expectMsgType[FundTransactionResponse].tx assert(txFunded1.txIn.nonEmpty) bitcoinClient.signPsbt(new Psbt(txFunded1), txFunded1.txIn.indices, Nil).pipeTo(sender.ref) - val signedTx1 = sender.expectMsgType[ProcessPsbtResponse].finalTx + val signedTx1 = sender.expectMsgType[ProcessPsbtResponse].finalTx_opt.toOption.get bitcoinClient.publishTransaction(signedTx1).pipeTo(sender.ref) assert(sender.expectMsgType[Failure].cause.getMessage.contains("min relay fee not met")) @@ -597,7 +580,7 @@ class BitcoinCoreClientSpec extends TestKitBaseClass with BitcoindService with A assert(txFunded2.txid != txFunded1.txid) txFunded1.txIn.foreach(txIn => assert(txFunded2.txIn.map(_.outPoint).contains(txIn.outPoint))) bitcoinClient.signPsbt(new Psbt(txFunded2), txFunded2.txIn.indices, Nil).pipeTo(sender.ref) - val signedTx2 = sender.expectMsgType[ProcessPsbtResponse].finalTx + val signedTx2 = sender.expectMsgType[ProcessPsbtResponse].finalTx_opt.toOption.get bitcoinClient.publishTransaction(signedTx2).pipeTo(sender.ref) sender.expectMsg(signedTx2.txid) awaitAssert({ @@ -609,7 +592,7 @@ class BitcoinCoreClientSpec extends TestKitBaseClass with BitcoindService with A test("unlock outpoints correctly") { val sender = TestProbe() val pubkeyScript = Script.write(Script.pay2wsh(Scripts.multiSig2of2(randomKey().publicKey, randomKey().publicKey))) - val bitcoinClient = makeBitcoinCoreClient + val bitcoinClient = makeBitcoinCoreClient() { // test #1: unlock outpoints that are actually locked @@ -649,14 +632,14 @@ class BitcoinCoreClientSpec extends TestKitBaseClass with BitcoindService with A import fr.acinq.bitcoin.scalacompat.KotlinUtils._ val sender = TestProbe() - val bitcoinClient = makeBitcoinCoreClient + val bitcoinClient = makeBitcoinCoreClient() val nonWalletKey = randomKey() val opts = FundTransactionOptions(TestConstants.feeratePerKw, changePosition = Some(1)) bitcoinClient.fundTransaction(Transaction(2, Nil, Seq(TxOut(250000 sat, Script.pay2wpkh(nonWalletKey.publicKey))), 0), opts).pipeTo(sender.ref) val fundedTx = sender.expectMsgType[FundTransactionResponse].tx bitcoinClient.signPsbt(new Psbt(fundedTx), fundedTx.txIn.indices, Nil).pipeTo(sender.ref) - val txToRemote = sender.expectMsgType[ProcessPsbtResponse].finalTx + val txToRemote = sender.expectMsgType[ProcessPsbtResponse].finalTx_opt.toOption.get bitcoinClient.publishTransaction(txToRemote).pipeTo(sender.ref) sender.expectMsg(txToRemote.txid) generateBlocks(1) @@ -674,11 +657,10 @@ class BitcoinCoreClientSpec extends TestKitBaseClass with BitcoindService with A bitcoinClient.signPsbt(new Psbt(txWithNonWalletInput), txWithNonWalletInput.txIn.indices.tail, Nil).pipeTo(sender.ref) val signTxResponse1 = sender.expectMsgType[ProcessPsbtResponse] assert(!signTxResponse1.complete) - signTxResponse1.extractPartiallySignedTx.txIn.tail.foreach(walletTxIn => assert(walletTxIn.witness.stack.nonEmpty)) + signTxResponse1.partiallySignedTx.txIn.tail.foreach(walletTxIn => assert(walletTxIn.witness.stack.nonEmpty)) // if the non-wallet inputs are signed, bitcoind signs the remaining wallet inputs. val nonWalletSig = Transaction.signInput(txWithNonWalletInput, 0, Script.pay2pkh(nonWalletKey.publicKey), bitcoin.SigHash.SIGHASH_ALL, txToRemote.txOut.head.amount, bitcoin.SigVersion.SIGVERSION_WITNESS_V0, nonWalletKey) - val nonWalletWitness = ScriptWitness(Seq(nonWalletSig, nonWalletKey.publicKey.value)) val txWithSignedNonWalletInput = txWithNonWalletInput.updateWitness(0, nonWalletWitness) val psbt = new Psbt(txWithSignedNonWalletInput) @@ -687,12 +669,7 @@ class BitcoinCoreClientSpec extends TestKitBaseClass with BitcoindService with A bitcoinClient.signPsbt(psbt1, txWithSignedNonWalletInput.txIn.indices.tail, Nil).pipeTo(sender.ref) val signTxResponse2 = sender.expectMsgType[ProcessPsbtResponse] assert(signTxResponse2.complete) - Transaction.correctlySpends(signTxResponse2.finalTx, txToRemote +: walletInputTxs, bitcoin.ScriptFlags.STANDARD_SCRIPT_VERIFY_FLAGS) - } - { - // bitcoind does not sign inputs that have already been confirmed. - //bitcoinClient.signPsbt(new Psbt(fundedTx), fundedTx.txIn.indices, Nil).pipeTo(sender.ref) - //sender.expectMsgType[Failure] + Transaction.correctlySpends(signTxResponse2.finalTx_opt.toOption.get, txToRemote +: walletInputTxs, bitcoin.ScriptFlags.STANDARD_SCRIPT_VERIFY_FLAGS) } { // bitcoind lets us double-spend ourselves. @@ -708,7 +685,7 @@ class BitcoinCoreClientSpec extends TestKitBaseClass with BitcoindService with A bitcoinClient.fundTransaction(Transaction(2, Nil, Seq(TxOut(125000 sat, Script.pay2wpkh(nonWalletKey.publicKey))), 0), opts).pipeTo(sender.ref) val fundedTx = sender.expectMsgType[FundTransactionResponse].tx bitcoinClient.signPsbt(new Psbt(fundedTx), fundedTx.txIn.indices, Nil).pipeTo(sender.ref) - val unconfirmedTx = sender.expectMsgType[ProcessPsbtResponse].finalTx + val unconfirmedTx = sender.expectMsgType[ProcessPsbtResponse].finalTx_opt.toOption.get bitcoinClient.publishTransaction(unconfirmedTx).pipeTo(sender.ref) sender.expectMsg(unconfirmedTx.txid) // bitcoind lets us use this unconfirmed non-wallet input. @@ -718,10 +695,9 @@ class BitcoinCoreClientSpec extends TestKitBaseClass with BitcoindService with A val nonWalletSig = Transaction.signInput(txWithUnconfirmedInput, 0, Script.pay2pkh(nonWalletKey.publicKey), bitcoin.SigHash.SIGHASH_ALL, unconfirmedTx.txOut.head.amount, bitcoin.SigVersion.SIGVERSION_WITNESS_V0, nonWalletKey) val nonWalletWitness = ScriptWitness(Seq(nonWalletSig, nonWalletKey.publicKey.value)) val txWithSignedUnconfirmedInput = txWithUnconfirmedInput.updateWitness(0, nonWalletWitness) - val previousTx = PreviousTx(Transactions.InputInfo(OutPoint(unconfirmedTx.txid, 0), unconfirmedTx.txOut.head, Script.pay2pkh(nonWalletKey.publicKey)), nonWalletWitness) val psbt = new Psbt(txWithSignedUnconfirmedInput) - val updated: Either[UpdateFailure, Psbt] = psbt.updateWitnessInput(psbt.getGlobal.getTx.txIn.get(0).outPoint, unconfirmedTx.txOut(0), null, fr.acinq.bitcoin.Script.pay2pkh(nonWalletKey.publicKey), SigHash.SIGHASH_ALL, psbt.getInput(0).getDerivationPaths) - val Right(psbt1) = updated.flatMap(_.finalizeWitnessInput(0, nonWalletWitness)) + val Right(psbt1) = psbt.updateWitnessInput(psbt.getGlobal.getTx.txIn.get(0).outPoint, unconfirmedTx.txOut(0), null, fr.acinq.bitcoin.Script.pay2pkh(nonWalletKey.publicKey), SigHash.SIGHASH_ALL, psbt.getInput(0).getDerivationPaths) + .flatMap(_.finalizeWitnessInput(0, nonWalletWitness)) bitcoinClient.signPsbt(psbt1, txWithSignedUnconfirmedInput.txIn.indices.tail, Nil).pipeTo(sender.ref) assert(sender.expectMsgType[ProcessPsbtResponse].complete) } @@ -731,7 +707,7 @@ class BitcoinCoreClientSpec extends TestKitBaseClass with BitcoindService with A import fr.acinq.bitcoin.scalacompat.KotlinUtils._ val sender = TestProbe() - val bitcoinClient = makeBitcoinCoreClient + val bitcoinClient = makeBitcoinCoreClient() val priv = randomKey() val noInputTx = Transaction(2, Nil, TxOut(6.btc.toSatoshi, Script.pay2wpkh(priv.publicKey)) :: Nil, 0) @@ -739,7 +715,7 @@ class BitcoinCoreClientSpec extends TestKitBaseClass with BitcoindService with A val fundTxResponse = sender.expectMsgType[FundTransactionResponse] val changePos = fundTxResponse.changePosition.get bitcoinClient.signPsbt(new Psbt(fundTxResponse.tx), fundTxResponse.tx.txIn.indices, Nil).pipeTo(sender.ref) - val tx = sender.expectMsgType[ProcessPsbtResponse].finalTx + val tx = sender.expectMsgType[ProcessPsbtResponse].finalTx_opt.toOption.get // we publish the tx a first time bitcoinClient.publishTransaction(tx).pipeTo(sender.ref) @@ -780,7 +756,7 @@ class BitcoinCoreClientSpec extends TestKitBaseClass with BitcoindService with A import fr.acinq.bitcoin.scalacompat.KotlinUtils._ val sender = TestProbe() - val bitcoinClient = makeBitcoinCoreClient + val bitcoinClient = makeBitcoinCoreClient() // that tx has inputs that don't exist val txWithUnknownInputs = Transaction.read("02000000000101b9e2a3f518fd74e696d258fed3c78c43f84504e76c99212e01cf225083619acf00000000000d0199800136b34b00000000001600145464ce1e5967773922506e285780339d72423244040047304402206795df1fd93c285d9028c384aacf28b43679f1c3f40215fd7bd1abbfb816ee5a022047a25b8c128e692d4717b6dd7b805aa24ecbbd20cfd664ab37a5096577d4a15d014730440220770f44121ed0e71ec4b482dded976f2febd7500dfd084108e07f3ce1e85ec7f5022025b32dc0d551c47136ce41bfb80f5a10de95c0babb22a3ae2d38e6688b32fcb20147522102c2662ab3e4fa18a141d3be3317c6ee134aff10e6cd0a91282a25bf75c0481ebc2102e952dd98d79aa796289fa438e4fdeb06ed8589ff2a0f032b0cfcb4d7b564bc3252aea58d1120") @@ -800,23 +776,23 @@ class BitcoinCoreClientSpec extends TestKitBaseClass with BitcoindService with A val signTxResponse = sender.expectMsgType[ProcessPsbtResponse] assert(signTxResponse.complete) - val txWithNoOutputs = signTxResponse.finalTx.copy(txOut = Nil) + val txWithNoOutputs = signTxResponse.finalTx_opt.toOption.get.copy(txOut = Nil) bitcoinClient.publishTransaction(txWithNoOutputs).pipeTo(sender.ref) sender.expectMsgType[Failure] bitcoinClient.getBlockHeight().pipeTo(sender.ref) val blockHeight = sender.expectMsgType[BlockHeight] - val txWithFutureCltv = signTxResponse.finalTx.copy(lockTime = blockHeight.toLong + 1) + val txWithFutureCltv = signTxResponse.finalTx_opt.toOption.get.copy(lockTime = blockHeight.toLong + 1) bitcoinClient.publishTransaction(txWithFutureCltv).pipeTo(sender.ref) sender.expectMsgType[Failure] - bitcoinClient.publishTransaction(signTxResponse.finalTx).pipeTo(sender.ref) - sender.expectMsg(signTxResponse.finalTx.txid) + bitcoinClient.publishTransaction(signTxResponse.finalTx_opt.toOption.get).pipeTo(sender.ref) + sender.expectMsg(signTxResponse.finalTx_opt.toOption.get.txid) } test("send and list transactions") { val sender = TestProbe() - val bitcoinClient = makeBitcoinCoreClient + val bitcoinClient = makeBitcoinCoreClient() bitcoinClient.onChainBalance().pipeTo(sender.ref) val initialBalance = sender.expectMsgType[OnChainBalance] @@ -852,16 +828,16 @@ class BitcoinCoreClientSpec extends TestKitBaseClass with BitcoindService with A val sender = TestProbe() val address = getNewAddress(sender) - val bitcoinClient = makeBitcoinCoreClient + val bitcoinClient = makeBitcoinCoreClient() def spendWalletTx(tx: Transaction, fees: Satoshi): Transaction = { val amount = tx.txOut.map(_.amount).sum - fees val unsignedTx = Transaction(version = 2, - txIn = tx.txOut.indices.map(i => TxIn(OutPoint(tx, i), Nil, fr.acinq.bitcoin.TxIn.SEQUENCE_FINAL)), + txIn = tx.txOut.indices.map(i => TxIn(OutPoint(tx, i), Nil, 0)), txOut = TxOut(amount, addressToPublicKeyScript(Block.RegtestGenesisBlock.hash, address).toOption.get) :: Nil, lockTime = 0) bitcoinClient.signPsbt(new Psbt(unsignedTx), unsignedTx.txIn.indices, Nil).pipeTo(sender.ref) - val signedTx = sender.expectMsgType[ProcessPsbtResponse].finalTx + val signedTx = sender.expectMsgType[ProcessPsbtResponse].finalTx_opt.toOption.get bitcoinClient.publishTransaction(signedTx).pipeTo(sender.ref) sender.expectMsg(signedTx.txid) signedTx @@ -897,20 +873,20 @@ class BitcoinCoreClientSpec extends TestKitBaseClass with BitcoindService with A test("abandon transaction") { val sender = TestProbe() - val bitcoinClient = makeBitcoinCoreClient + val bitcoinClient = makeBitcoinCoreClient() // Broadcast a wallet transaction. val opts = FundTransactionOptions(TestConstants.feeratePerKw, changePosition = Some(1)) bitcoinClient.fundTransaction(Transaction(2, Nil, Seq(TxOut(250000 sat, Script.pay2wpkh(randomKey().publicKey))), 0), opts).pipeTo(sender.ref) val fundedTx1 = sender.expectMsgType[FundTransactionResponse].tx - signTransaction(bitcoinClient, fundedTx1, Nil).pipeTo(sender.ref) + signTransaction(bitcoinClient, fundedTx1).pipeTo(sender.ref) val signedTx1 = sender.expectMsgType[SignTransactionResponse].tx bitcoinClient.publishTransaction(signedTx1).pipeTo(sender.ref) sender.expectMsg(signedTx1.txid) // Double-spend that transaction. val fundedTx2 = fundedTx1.copy(txOut = TxOut(200000 sat, Script.pay2wpkh(randomKey().publicKey)) +: fundedTx1.txOut.tail) - signTransaction(bitcoinClient, fundedTx2, Nil).pipeTo(sender.ref) + signTransaction(bitcoinClient, fundedTx2).pipeTo(sender.ref) val signedTx2 = sender.expectMsgType[SignTransactionResponse].tx assert(signedTx2.txid != signedTx1.txid) bitcoinClient.publishTransaction(signedTx2).pipeTo(sender.ref) @@ -935,7 +911,7 @@ class BitcoinCoreClientSpec extends TestKitBaseClass with BitcoindService with A test("bump transaction fees with child-pays-for-parent (single tx)") { val sender = TestProbe() - val bitcoinClient = makeBitcoinCoreClient + val bitcoinClient = makeBitcoinCoreClient() val tx = sendToAddress(getNewAddress(sender), 150_000 sat) assert(tx.txOut.length == 2) // there must be a change output val changeOutput = if (tx.txOut.head.amount == 150_000.sat) 1 else 0 @@ -962,7 +938,7 @@ class BitcoinCoreClientSpec extends TestKitBaseClass with BitcoindService with A test("bump transaction fees with child-pays-for-parent (small package)") { val sender = TestProbe() - val bitcoinClient = makeBitcoinCoreClient + val bitcoinClient = makeBitcoinCoreClient() val fundingFeerate = FeeratePerKw(1000 sat) val remoteFundingPrivKey = randomKey() @@ -973,7 +949,7 @@ class BitcoinCoreClientSpec extends TestKitBaseClass with BitcoindService with A bitcoinClient.fundTransaction(txNotFunded, FundTransactionOptions(fundingFeerate, changePosition = Some(1))).pipeTo(sender.ref) val fundTxResponse = sender.expectMsgType[FundTransactionResponse] assert(fundTxResponse.changePosition.contains(1)) - signTransaction(bitcoinClient, fundTxResponse.tx, Nil).pipeTo(sender.ref) + signTransaction(bitcoinClient, fundTxResponse.tx).pipeTo(sender.ref) val signTxResponse = sender.expectMsgType[SignTransactionResponse] assert(signTxResponse.complete) bitcoinClient.publishTransaction(signTxResponse.tx).pipeTo(sender.ref) @@ -1018,7 +994,7 @@ class BitcoinCoreClientSpec extends TestKitBaseClass with BitcoindService with A test("bump transaction fees with child-pays-for-parent (complex package)") { val sender = TestProbe() - val bitcoinClient = makeBitcoinCoreClient + val bitcoinClient = makeBitcoinCoreClient() val currentFeerate = FeeratePerKw(500 sat) // We create two separate trees of transactions that will be bumped together: @@ -1042,7 +1018,7 @@ class BitcoinCoreClientSpec extends TestKitBaseClass with BitcoindService with A val txNotFunded = Transaction(2, txIn, txOut, 0) bitcoinClient.fundTransaction(txNotFunded, FundTransactionOptions(currentFeerate, changePosition = Some(txOut.length))).pipeTo(sender.ref) val fundTxResponse = sender.expectMsgType[FundTransactionResponse] - signTransaction(bitcoinClient, fundTxResponse.tx, Nil).pipeTo(sender.ref) + signTransaction(bitcoinClient, fundTxResponse.tx).pipeTo(sender.ref) val signTxResponse = sender.expectMsgType[SignTransactionResponse] assert(signTxResponse.complete) bitcoinClient.publishTransaction(signTxResponse.tx).pipeTo(sender.ref) @@ -1113,7 +1089,7 @@ class BitcoinCoreClientSpec extends TestKitBaseClass with BitcoindService with A test("cannot bump transaction fees (unknown transaction)") { val sender = TestProbe() - val bitcoinClient = makeBitcoinCoreClient + val bitcoinClient = makeBitcoinCoreClient() bitcoinClient.cpfp(Set(OutPoint(randomBytes32(), 0), OutPoint(randomBytes32(), 3)), FeeratePerKw(1500 sat)).pipeTo(sender.ref) val failure = sender.expectMsgType[Failure] assert(failure.cause.getMessage.contains("some transactions could not be found")) @@ -1121,7 +1097,7 @@ class BitcoinCoreClientSpec extends TestKitBaseClass with BitcoindService with A test("cannot bump transaction fees (invalid outpoint index)") { val sender = TestProbe() - val bitcoinClient = makeBitcoinCoreClient + val bitcoinClient = makeBitcoinCoreClient() val tx = sendToAddress(getNewAddress(sender), 150_000 sat) assert(tx.txOut.length == 2) // there must be a change output bitcoinClient.getMempoolTx(tx.txid).pipeTo(sender.ref) @@ -1136,7 +1112,7 @@ class BitcoinCoreClientSpec extends TestKitBaseClass with BitcoindService with A test("cannot bump transaction fees (transaction already confirmed)") { val sender = TestProbe() - val bitcoinClient = makeBitcoinCoreClient + val bitcoinClient = makeBitcoinCoreClient() val tx = sendToAddress(getNewAddress(sender), 45_000 sat) generateBlocks(1) @@ -1147,23 +1123,23 @@ class BitcoinCoreClientSpec extends TestKitBaseClass with BitcoindService with A test("cannot bump transaction fees (non-wallet input)") { val sender = TestProbe() - val bitcoinClient = makeBitcoinCoreClient + val bitcoinClient = makeBitcoinCoreClient() val txNotFunded = Transaction(2, Nil, TxOut(50_000 sat, Script.pay2wpkh(randomKey().publicKey)) :: Nil, 0) bitcoinClient.fundTransaction(txNotFunded, FundTransactionOptions(FeeratePerKw(1000 sat), changePosition = Some(1))).pipeTo(sender.ref) val fundTxResponse = sender.expectMsgType[FundTransactionResponse] - signTransaction(bitcoinClient, fundTxResponse.tx, Nil).pipeTo(sender.ref) + signTransaction(bitcoinClient, fundTxResponse.tx).pipeTo(sender.ref) val signTxResponse = sender.expectMsgType[SignTransactionResponse] bitcoinClient.publishTransaction(signTxResponse.tx).pipeTo(sender.ref) sender.expectMsg(signTxResponse.tx.txid) bitcoinClient.cpfp(Set(OutPoint(signTxResponse.tx, 0)), FeeratePerKw(1500 sat)).pipeTo(sender.ref) val failure = sender.expectMsgType[Failure] - assert(failure.cause.getMessage.contains("some inputs don't belong to our wallet")) + assert(failure.cause.getMessage.contains("tx signing failed")) } test("cannot bump transaction fees (amount too low)") { val sender = TestProbe() - val bitcoinClient = makeBitcoinCoreClient + val bitcoinClient = makeBitcoinCoreClient() val tx = sendToAddress(getNewAddress(sender), 2500 sat) val outputIndex = if (tx.txOut.head.amount == 2500.sat) 0 else 1 bitcoinClient.cpfp(Set(OutPoint(tx, outputIndex)), FeeratePerKw(50_000 sat)).pipeTo(sender.ref) @@ -1173,7 +1149,7 @@ class BitcoinCoreClientSpec extends TestKitBaseClass with BitcoindService with A test("detect if tx has been double-spent") { val sender = TestProbe() - val bitcoinClient = makeBitcoinCoreClient + val bitcoinClient = makeBitcoinCoreClient() // first let's create a tx val noInputTx1 = Transaction(2, Nil, Seq(TxOut(500_000 sat, Script.pay2wpkh(randomKey().publicKey))), 0) @@ -1211,7 +1187,7 @@ class BitcoinCoreClientSpec extends TestKitBaseClass with BitcoindService with A test("detect if tx has been double-spent (with unconfirmed inputs)") { val sender = TestProbe() - val bitcoinClient = makeBitcoinCoreClient + val bitcoinClient = makeBitcoinCoreClient() val priv = randomKey() // Let's create one confirmed and one unconfirmed utxo. @@ -1273,7 +1249,7 @@ class BitcoinCoreClientSpec extends TestKitBaseClass with BitcoindService with A test("find spending transaction of a given output") { val sender = TestProbe() - val bitcoinClient = makeBitcoinCoreClient + val bitcoinClient = makeBitcoinCoreClient() bitcoinClient.getBlockHeight().pipeTo(sender.ref) val blockHeight = sender.expectMsgType[BlockHeight] @@ -1321,9 +1297,9 @@ class BitcoinCoreClientSpec extends TestKitBaseClass with BitcoindService with A test("get pubkey for p2wpkh receive address") { val sender = TestProbe() - val bitcoinClient = makeBitcoinCoreClient + val bitcoinClient = makeBitcoinCoreClient() - // eclair onchain key nmanager does not yet support taproot descriptors + // eclair on-chain key manager does not yet support taproot descriptors bitcoinClient.getReceiveAddress().pipeTo(sender.ref) val defaultAddress = sender.expectMsgType[String] val decoded = Bech32.decodeWitnessAddress(defaultAddress) @@ -1344,7 +1320,7 @@ class BitcoinCoreClientSpec extends TestKitBaseClass with BitcoindService with A test("generate segwit change outputs") { val sender = TestProbe() - val bitcoinClient = makeBitcoinCoreClient + val bitcoinClient = makeBitcoinCoreClient() // Even when we pay a legacy address, our change output must use segwit, otherwise it won't be usable for lightning channels. val pubKey = randomKey().publicKey @@ -1367,7 +1343,7 @@ class BitcoinCoreClientSpec extends TestKitBaseClass with BitcoindService with A val wallet = new BitcoinCoreClient(createWallet("mempool_eviction", sender)) wallet.getP2wpkhPubkey().pipeTo(sender.ref) val walletPubKey = sender.expectMsgType[PublicKey] - val miner = makeBitcoinCoreClient + val miner = makeBitcoinCoreClient() miner.getP2wpkhPubkey().pipeTo(sender.ref) val nonWalletPubKey = sender.expectMsgType[PublicKey] // We use a large input script to be able to fill the mempool with a few transactions. @@ -1381,7 +1357,7 @@ class BitcoinCoreClientSpec extends TestKitBaseClass with BitcoindService with A val txNotFunded = Transaction(2, Nil, mainOutput +: outputsWithLargeScript, 0) miner.fundTransaction(txNotFunded, FundTransactionOptions(FeeratePerKw(500 sat), changePosition = Some(outputs.length))).pipeTo(sender.ref) val fundedTx = sender.expectMsgType[FundTransactionResponse].tx - signTransaction(miner, fundedTx, allowIncomplete = false).pipeTo(sender.ref) + signTransaction(miner, fundedTx).pipeTo(sender.ref) val signedTx = sender.expectMsgType[SignTransactionResponse].tx miner.publishTransaction(signedTx).pipeTo(sender.ref) sender.expectMsg(signedTx.txid) @@ -1400,7 +1376,7 @@ class BitcoinCoreClientSpec extends TestKitBaseClass with BitcoindService with A psbt = psbt.finalizeWitnessInput(i, ScriptWitness(Seq(ByteVector(1), bigInputScript))).getRight }) wallet.signPsbt(psbt, Seq(0), Nil).pipeTo(sender.ref) - val signedTx = sender.expectMsgType[ProcessPsbtResponse].finalTx + val signedTx = sender.expectMsgType[ProcessPsbtResponse].finalTx_opt.toOption.get assert(390_000 <= signedTx.weight() && signedTx.weight() <= 400_000) // standard transactions cannot exceed 400 000 WU wallet.publishTransaction(signedTx).pipeTo(sender.ref) sender.expectMsg(signedTx.txid) @@ -1429,41 +1405,42 @@ class BitcoinCoreClientSpec extends TestKitBaseClass with BitcoindService with A class BitcoinCoreClientWithEclairSignerSpec extends BitcoinCoreClientSpec { override def useEclairSigner = true + private def createWallet(seed: ByteVector): (BitcoinCoreClient, LocalOnChainKeyManager) = { + val name = s"eclair_${seed.toString()}" + val onChainKeyManager = new LocalOnChainKeyManager(name, seed, TimestampSecond.now(), Block.RegtestGenesisBlock.hash) + val jsonRpcClient = new BasicBitcoinJsonRPCClient(rpcAuthMethod = bitcoinrpcauthmethod, host = "localhost", port = bitcoindRpcPort, wallet = Some(name)) + (new BitcoinCoreClient(jsonRpcClient, Some(onChainKeyManager)), onChainKeyManager) + } + test("wallets managed by eclair implement BIP84") { val sender = TestProbe() val entropy = randomBytes32() - val hex = entropy.toString() - val mnemmonics = MnemonicCode.toMnemonics(entropy) - val seed = MnemonicCode.toSeed(mnemmonics, "") + val seed = MnemonicCode.toSeed(MnemonicCode.toMnemonics(entropy), "") val master = DeterministicWallet.generate(seed) - - val onchainKeyManager = new LocalOnchainKeyManager(s"eclair_$hex", seed, TimestampSecond.now(), Block.RegtestGenesisBlock.hash) - val jsonRpcClient = new BasicBitcoinJsonRPCClient(rpcAuthMethod = bitcoinrpcauthmethod, host = "localhost", port = bitcoindRpcPort, wallet = Some(s"eclair_$hex")) - val wallet1 = new BitcoinCoreClient(jsonRpcClient, Some(onchainKeyManager)) - wallet1.createEclairBackedWallet().pipeTo(sender.ref) + val (wallet, keyManager) = createWallet(seed) + keyManager.createWallet(wallet.rpcClient).pipeTo(sender.ref) sender.expectMsg(true) // this account xpub can be used to create a watch-only wallet val accountXPub = DeterministicWallet.encode( DeterministicWallet.publicKey(DeterministicWallet.derivePrivateKey(master, DeterministicWallet.KeyPath("m/84'/1'/0'"))), DeterministicWallet.vpub) - assert(onchainKeyManager.getOnchainMasterPubKey(0) == accountXPub) - + assert(wallet.onChainKeyManager_opt.get.masterPubKey(0) == accountXPub) def getBip32Path(address: String): DeterministicWallet.KeyPath = { - wallet1.rpcClient.invoke("getaddressinfo", address).pipeTo(sender.ref) - val JString(bip32path) = (sender.expectMsgType[JValue] \ "hdkeypath") + wallet.rpcClient.invoke("getaddressinfo", address).pipeTo(sender.ref) + val JString(bip32path) = sender.expectMsgType[JValue] \ "hdkeypath" DeterministicWallet.KeyPath(bip32path) } (0 to 10).foreach { _ => - wallet1.getReceiveAddress().pipeTo(sender.ref) + wallet.getReceiveAddress().pipeTo(sender.ref) val address = sender.expectMsgType[String] val bip32path = getBip32Path(address) assert(bip32path.path.length == 5 && bip32path.toString().startsWith("m/84'/1'/0'/0")) assert(computeBIP84Address(DeterministicWallet.derivePrivateKey(master, bip32path).publicKey, Block.RegtestGenesisBlock.hash) == address) - wallet1.getP2wpkhPubkeyHashForChange().pipeTo(sender.ref) + wallet.getP2wpkhPubkeyHashForChange().pipeTo(sender.ref) val Right(changeAddress) = addressFromPublicKeyScript(Block.RegtestGenesisBlock.hash, Script.pay2wpkh(sender.expectMsgType[ByteVector])) val bip32ChangePath = getBip32Path(changeAddress) assert(bip32ChangePath.path.length == 5 && bip32ChangePath.toString().startsWith("m/84'/1'/0'/1")) @@ -1471,30 +1448,26 @@ class BitcoinCoreClientWithEclairSignerSpec extends BitcoinCoreClientSpec { } } - test("use eclair to manage onchain keys") { + test("use eclair to manage on-chain keys") { val sender = TestProbe() (1 to 10).foreach { _ => - val seed = randomBytes32() - val hex = seed.toString() - val onchainKeyManager = new LocalOnchainKeyManager(s"eclair_$hex", seed, TimestampSecond.now(), Block.RegtestGenesisBlock.hash) - val jsonRpcClient = new BasicBitcoinJsonRPCClient(rpcAuthMethod = bitcoinrpcauthmethod, host = "localhost", port = bitcoindRpcPort, wallet = Some(s"eclair_$hex")) - val wallet1 = new BitcoinCoreClient(jsonRpcClient, Some(onchainKeyManager)) - wallet1.createEclairBackedWallet().pipeTo(sender.ref) + val (wallet, keyManager) = createWallet(randomBytes32()) + keyManager.createWallet(wallet.rpcClient).pipeTo(sender.ref) sender.expectMsg(true) - wallet1.getReceiveAddress().pipeTo(sender.ref) + wallet.getReceiveAddress().pipeTo(sender.ref) val address = sender.expectMsgType[String] - // we can send to an onchain address if eclair signs the transactions + // we can send to an on-chain address if eclair signs the transactions sendToAddress(address, 100_0000.sat) generateBlocks(1) // but bitcoin core's sendtoaddress RPC call will fail because wallets uses an external signer - wallet1.sendToAddress(address, 50_000.sat, 3).pipeTo(sender.ref) + wallet.sendToAddress(address, 50_000.sat, 3).pipeTo(sender.ref) val error = sender.expectMsgType[Failure] assert(error.cause.getMessage.contains("Private keys are disabled for this wallet")) - wallet1.sendToPubkeyScript(addressToPublicKeyScript(Block.RegtestGenesisBlock.hash, address).toOption.get, 50_000.sat, FeeratePerKw(FeeratePerByte(5.sat))).pipeTo(sender.ref) + wallet.sendToPubkeyScript(addressToPublicKeyScript(Block.RegtestGenesisBlock.hash, address).toOption.get, 50_000.sat, FeeratePerKw(FeeratePerByte(5.sat))).pipeTo(sender.ref) sender.expectMsgType[ByteVector32] } } diff --git a/eclair-core/src/test/scala/fr/acinq/eclair/blockchain/bitcoind/BitcoindService.scala b/eclair-core/src/test/scala/fr/acinq/eclair/blockchain/bitcoind/BitcoindService.scala index 9d4c2426b7..4aabf798be 100644 --- a/eclair-core/src/test/scala/fr/acinq/eclair/blockchain/bitcoind/BitcoindService.scala +++ b/eclair-core/src/test/scala/fr/acinq/eclair/blockchain/bitcoind/BitcoindService.scala @@ -22,11 +22,10 @@ import akka.testkit.{TestKitBase, TestProbe} import fr.acinq.bitcoin.psbt.Psbt import fr.acinq.bitcoin.scalacompat.Crypto.PrivateKey import fr.acinq.bitcoin.scalacompat.{Block, Btc, BtcAmount, MilliBtc, MnemonicCode, Satoshi, Transaction, TxOut, addressToPublicKeyScript, computeP2WpkhAddress} -import fr.acinq.eclair.blockchain.bitcoind.rpc.BitcoinCoreClient.PreviousTx import fr.acinq.eclair.blockchain.bitcoind.rpc.BitcoinJsonRPCAuthMethod.{SafeCookie, UserPassword} import fr.acinq.eclair.blockchain.bitcoind.rpc.{BasicBitcoinJsonRPCClient, BitcoinCoreClient, BitcoinJsonRPCAuthMethod, BitcoinJsonRPCClient} import fr.acinq.eclair.blockchain.fee.{FeeratePerByte, FeeratePerKB, FeeratePerKw} -import fr.acinq.eclair.crypto.keymanager.LocalOnchainKeyManager +import fr.acinq.eclair.crypto.keymanager.LocalOnChainKeyManager import fr.acinq.eclair.integration.IntegrationSpec import fr.acinq.eclair.{BlockHeight, TestUtils, TimestampSecond, randomKey} import grizzled.slf4j.Logging @@ -87,9 +86,7 @@ trait BitcoindService extends Logging { | } |} |""".stripMargin - val onchainKeyManager = { - new LocalOnchainKeyManager("eclair", MnemonicCode.toSeed(mnemonics, passphrase), TimestampSecond.now(), Block.RegtestGenesisBlock.hash) - } + val onChainKeyManager = new LocalOnChainKeyManager("eclair", MnemonicCode.toSeed(mnemonics, passphrase), TimestampSecond.now(), Block.RegtestGenesisBlock.hash) def startBitcoind(useCookie: Boolean = false, defaultAddressType_opt: Option[String] = None, @@ -136,7 +133,7 @@ trait BitcoindService extends Logging { })) } - def makeBitcoinCoreClient: BitcoinCoreClient = new BitcoinCoreClient(bitcoinrpcclient, Some(onchainKeyManager)) + def makeBitcoinCoreClient(): BitcoinCoreClient = new BitcoinCoreClient(bitcoinrpcclient, if (useEclairSigner) Some(onChainKeyManager) else None) def stopBitcoind(): Unit = { // gracefully stopping bitcoin will make it store its state cleanly to disk, which is good for later debugging @@ -174,7 +171,7 @@ trait BitcoindService extends Logging { val sender = TestProbe() waitForBitcoindUp(sender) if (useEclairSigner) { - makeBitcoinCoreClient.createEclairBackedWallet().pipeTo(sender.ref) + onChainKeyManager.createWallet(bitcoinrpcclient).pipeTo(sender.ref) sender.expectMsg(true) } else { sender.send(bitcoincli, BitcoinReq("createwallet", defaultWallet)) @@ -185,6 +182,7 @@ trait BitcoindService extends Logging { awaitCond(currentBlockHeight(sender) >= BlockHeight(150), max = 3 minutes, interval = 2 second) } + /** Generate blocks to a given address, or to our wallet if no address is provided. */ def generateBlocks(blockCount: Int, address: Option[String] = None, timeout: FiniteDuration = 10 seconds)(implicit system: ActorSystem): Unit = { val sender = TestProbe() val addressToUse = address match { @@ -237,11 +235,11 @@ trait BitcoindService extends Logging { } val probe = TestProbe() val tx = Transaction(version = 2, Nil, TxOut(amountSat, addressToPublicKeyScript(Block.RegtestGenesisBlock.hash, address).toOption.get) :: Nil, lockTime = 0) - val client = makeBitcoinCoreClient + val client = makeBitcoinCoreClient() val f = for { - funded <- client.fundTransaction(tx, FeeratePerKw(FeeratePerByte(Satoshi(10))), true) + funded <- client.fundTransaction(tx, FeeratePerKw(FeeratePerByte(Satoshi(10))), replaceable = true) signed <- client.signPsbt(new Psbt(funded.tx), funded.tx.txIn.indices, Nil) - txid <- client.publishTransaction(signed.finalTx) + txid <- client.publishTransaction(signed.finalTx_opt.toOption.get) tx <- client.getTransaction(txid) } yield tx f.pipeTo(probe.ref) @@ -262,13 +260,9 @@ trait BitcoindService extends Logging { Transaction.read(rawTx) } - def signTransaction(client: BitcoinCoreClient, tx: Transaction): Future[SignTransactionResponse] = signTransaction(client, tx, Nil) - - def signTransaction(client: BitcoinCoreClient, tx: Transaction, allowIncomplete: Boolean): Future[SignTransactionResponse] = signTransaction(client, tx, Nil, allowIncomplete) - - def signTransaction(client: BitcoinCoreClient, tx: Transaction, previousTxs: Seq[PreviousTx], allowIncomplete: Boolean = false): Future[SignTransactionResponse] = { + def signTransaction(client: BitcoinCoreClient, tx: Transaction): Future[SignTransactionResponse] = { import fr.acinq.bitcoin.scalacompat.KotlinUtils._ - client.signPsbt(new Psbt(tx), tx.txIn.indices, Nil).map(p => SignTransactionResponse(p.extractFinalTx.getOrElse(p.extractPartiallySignedTx), p.extractFinalTx.isRight)) + client.signPsbt(new Psbt(tx), tx.txIn.indices, Nil).map(p => SignTransactionResponse(p.finalTx_opt.getOrElse(p.partiallySignedTx), p.finalTx_opt.isRight)) } } diff --git a/eclair-core/src/test/scala/fr/acinq/eclair/channel/InteractiveTxBuilderSpec.scala b/eclair-core/src/test/scala/fr/acinq/eclair/channel/InteractiveTxBuilderSpec.scala index 8641e2bec0..ff6ddec1d4 100644 --- a/eclair-core/src/test/scala/fr/acinq/eclair/channel/InteractiveTxBuilderSpec.scala +++ b/eclair-core/src/test/scala/fr/acinq/eclair/channel/InteractiveTxBuilderSpec.scala @@ -24,9 +24,9 @@ import com.softwaremill.quicklens.{ModifyPimp, QuicklensAt} import fr.acinq.bitcoin.psbt.Psbt import fr.acinq.bitcoin.scalacompat.Crypto.PublicKey import fr.acinq.bitcoin.scalacompat.{Block, ByteVector32, ByteVector64, OP_1, OutPoint, Satoshi, SatoshiLong, Script, ScriptWitness, Transaction, TxOut, addressToPublicKeyScript} -import fr.acinq.eclair.blockchain.OnChainWallet.FundTransactionResponse +import fr.acinq.eclair.blockchain.OnChainWallet.{FundTransactionResponse, ProcessPsbtResponse} import fr.acinq.eclair.blockchain.bitcoind.BitcoindService -import fr.acinq.eclair.blockchain.bitcoind.rpc.BitcoinCoreClient.{MempoolTx, ProcessPsbtResponse, Utxo} +import fr.acinq.eclair.blockchain.bitcoind.rpc.BitcoinCoreClient.{MempoolTx, Utxo} import fr.acinq.eclair.blockchain.bitcoind.rpc.{BitcoinCoreClient, BitcoinJsonRPCClient} import fr.acinq.eclair.blockchain.fee.{FeeratePerByte, FeeratePerKw} import fr.acinq.eclair.blockchain.{OnChainWallet, SingleKeyOnChainWallet} @@ -63,11 +63,11 @@ class InteractiveTxBuilderSpec extends TestKitBaseClass with AnyFunSuiteLike wit wallet.getReceiveAddress().pipeTo(probe.ref) val walletAddress = probe.expectMsgType[String] val tx = Transaction(version = 2, Nil, TxOut(amount, addressToPublicKeyScript(Block.RegtestGenesisBlock.hash, walletAddress).toOption.get) :: Nil, lockTime = 0) - val client = makeBitcoinCoreClient + val client = makeBitcoinCoreClient() val f = for { - funded <- client.fundTransaction(tx, FeeratePerKw(FeeratePerByte(10.sat)), true) + funded <- client.fundTransaction(tx, FeeratePerKw(FeeratePerByte(10.sat)), replaceable = true) signed <- client.signPsbt(new Psbt(funded.tx), funded.tx.txIn.indices, Nil) - txid <- client.publishTransaction(signed.finalTx) + txid <- client.publishTransaction(signed.finalTx_opt.toOption.get) } yield txid f.pipeTo(probe.ref) probe.expectMsgType[ByteVector32] @@ -1004,11 +1004,11 @@ class InteractiveTxBuilderSpec extends TestKitBaseClass with AnyFunSuiteLike wit walletA.getP2wpkhPubkey().pipeTo(probe.ref) val publicKey = probe.expectMsgType[PublicKey] val tx = Transaction(2, Nil, TxOut(100_000 sat, Script.pay2wpkh(publicKey)) +: (1 to 2500).map(_ => TxOut(5000 sat, Script.pay2wpkh(randomKey().publicKey))), 0) - val minerWallet = makeBitcoinCoreClient + val minerWallet = makeBitcoinCoreClient() minerWallet.fundTransaction(tx, FeeratePerKw(500 sat), replaceable = true).pipeTo(probe.ref) val unsignedTx = probe.expectMsgType[FundTransactionResponse].tx minerWallet.signPsbt(new Psbt(unsignedTx), unsignedTx.txIn.indices, Nil).pipeTo(probe.ref) - val signedTx = probe.expectMsgType[ProcessPsbtResponse].finalTx + val Right(signedTx) = probe.expectMsgType[ProcessPsbtResponse].finalTx_opt assert(Transaction.write(signedTx).length >= 65_000) minerWallet.publishTransaction(signedTx).pipeTo(probe.ref) probe.expectMsgType[ByteVector32] diff --git a/eclair-core/src/test/scala/fr/acinq/eclair/channel/publish/ReplaceableTxPublisherSpec.scala b/eclair-core/src/test/scala/fr/acinq/eclair/channel/publish/ReplaceableTxPublisherSpec.scala index 507d3a248d..6245b5f5db 100644 --- a/eclair-core/src/test/scala/fr/acinq/eclair/channel/publish/ReplaceableTxPublisherSpec.scala +++ b/eclair-core/src/test/scala/fr/acinq/eclair/channel/publish/ReplaceableTxPublisherSpec.scala @@ -36,7 +36,7 @@ import fr.acinq.eclair.channel.publish.ReplaceableTxPublisher.{Publish, Stop, Up import fr.acinq.eclair.channel.publish.TxPublisher.TxRejectedReason._ import fr.acinq.eclair.channel.publish.TxPublisher._ import fr.acinq.eclair.channel.states.{ChannelStateTestsBase, ChannelStateTestsTags} -import fr.acinq.eclair.crypto.keymanager.LocalOnchainKeyManager +import fr.acinq.eclair.crypto.keymanager.LocalOnChainKeyManager import fr.acinq.eclair.transactions.Transactions import fr.acinq.eclair.transactions.Transactions._ import fr.acinq.eclair.wire.protocol.{CommitSig, RevokeAndAck} @@ -1655,7 +1655,7 @@ class ReplaceableTxPublisherWithEclairSignerSpec extends ReplaceableTxPublisherS // we use the wallet name as a passphrase to make sure we get a new empty wallet val entropy = ByteVector.fromValidHex("01" * 32) val seed = MnemonicCode.toSeed(MnemonicCode.toMnemonics(entropy), walletName) - val keyManager = new LocalOnchainKeyManager(walletName, seed, TimestampSecond.now(), Block.RegtestGenesisBlock.hash) + val keyManager = new LocalOnChainKeyManager(walletName, seed, TimestampSecond.now(), Block.RegtestGenesisBlock.hash) val walletRpcClient = new BasicBitcoinJsonRPCClient(rpcAuthMethod = bitcoinrpcauthmethod, host = "localhost", port = bitcoindRpcPort, wallet = Some(walletName)) val walletClient = new BitcoinCoreClient(walletRpcClient, Some(keyManager)) with OnchainPubkeyCache { lazy val pubkey = { @@ -1665,9 +1665,9 @@ class ReplaceableTxPublisherWithEclairSignerSpec extends ReplaceableTxPublisherS override def getP2wpkhPubkey(renew: Boolean): PublicKey = pubkey } - walletClient.createEclairBackedWallet().pipeTo(probe.ref) + keyManager.createWallet(walletRpcClient).pipeTo(probe.ref) probe.expectMsg(true) - + (walletRpcClient, walletClient) } } \ No newline at end of file diff --git a/eclair-core/src/test/scala/fr/acinq/eclair/crypto/keymanager/LocalOnchainKeyManagerSpec.scala b/eclair-core/src/test/scala/fr/acinq/eclair/crypto/keymanager/LocalOnChainKeyManagerSpec.scala similarity index 83% rename from eclair-core/src/test/scala/fr/acinq/eclair/crypto/keymanager/LocalOnchainKeyManagerSpec.scala rename to eclair-core/src/test/scala/fr/acinq/eclair/crypto/keymanager/LocalOnChainKeyManagerSpec.scala index 0c5c7c26fb..353d34c3da 100644 --- a/eclair-core/src/test/scala/fr/acinq/eclair/crypto/keymanager/LocalOnchainKeyManagerSpec.scala +++ b/eclair-core/src/test/scala/fr/acinq/eclair/crypto/keymanager/LocalOnChainKeyManagerSpec.scala @@ -11,17 +11,17 @@ import java.util.Base64 import scala.jdk.CollectionConverters.SeqHasAsJava import scala.util.{Failure, Success} -class LocalOnchainKeyManagerSpec extends AnyFunSuite { +class LocalOnChainKeyManagerSpec extends AnyFunSuite { test("sign psbt (non-reg test)") { val entropy = ByteVector.fromValidHex("01" * 32) val seed = MnemonicCode.toSeed(MnemonicCode.toMnemonics(entropy), "") - val onchainKeyManager = new LocalOnchainKeyManager("eclair", seed, TimestampSecond.now(), Block.TestnetGenesisBlock.hash) + val onChainKeyManager = new LocalOnChainKeyManager("eclair", seed, TimestampSecond.now(), Block.TestnetGenesisBlock.hash) // data generated by bitcoin core on regtest val psbt = Psbt.read( Base64.getDecoder.decode("cHNidP8BAHECAAAAAfZo4nGIyTg77MFmEBkQH1Au3Jl8vzB2WWQGGz/MbyssAAAAAAD9////ArAHPgUAAAAAFgAU6j9yVvLg66Zu3GM/xHbmXT0yvyiAlpgAAAAAABYAFODscQh3N7lmDYyV5yrHpGL2Zd4JAAAAAAABAH0CAAAAAaNdmqUNlziIjSaif3JUcvJWdyF0U5bYq13NMe+LbaBZAAAAAAD9////AjSp1gUAAAAAFgAUjfFMfBg8ulo/874n3+0ode7ka0BAQg8AAAAAACIAIPUn/XU17DfnvDkj8gn2twG3jtr2Z7sthy9K2MPTdYkaAAAAAAEBHzSp1gUAAAAAFgAUjfFMfBg8ulo/874n3+0ode7ka0AiBgM+PDdyxsVisa66SyBxiUvhEam8lEP64yujvVsEcGaqIxgPCfOBVAAAgAEAAIAAAACAAQAAAAMAAAAAIgIDWmAhb/sCV9+HjwFpPuy2TyEBi/Y11wrEHZUihe3N80EYDwnzgVQAAIABAACAAAAAgAEAAAAFAAAAAAA=") ).getRight - val Success(psbt1) = onchainKeyManager.signPsbt(psbt, psbt.getInputs.toArray().indices, Seq(0)) + val Success(psbt1) = onChainKeyManager.sign(psbt, psbt.getInputs.toArray().indices, Seq(0)) val tx = psbt1.extract() assert(tx.isRight) } @@ -30,19 +30,18 @@ class LocalOnchainKeyManagerSpec extends AnyFunSuite { import fr.acinq.bitcoin.scalacompat.KotlinUtils._ val seed = ByteVector.fromValidHex("01" * 32) - val onchainKeyManager = new LocalOnchainKeyManager("eclair", seed, TimestampSecond.now(), Block.TestnetGenesisBlock.hash) + val onChainKeyManager = new LocalOnChainKeyManager("eclair", seed, TimestampSecond.now(), Block.TestnetGenesisBlock.hash) // create a watch-only BIP84 wallet from our key manager xpub - val (_, accountPub) = DeterministicWallet.ExtendedPublicKey.decode(onchainKeyManager.getOnchainMasterPubKey(0)) + val (_, accountPub) = DeterministicWallet.ExtendedPublicKey.decode(onChainKeyManager.masterPubKey(0)) val mainPub = DeterministicWallet.derivePublicKey(accountPub, 0) - val changePub = DeterministicWallet.derivePublicKey(accountPub, 1) def getPublicKey(index: Long) = DeterministicWallet.derivePublicKey(mainPub, index).publicKey val utxos = Seq( - Transaction(version = 2, txIn = Nil, txOut = TxOut(Satoshi(1000_000), Script.pay2wpkh(getPublicKey(0))) :: Nil, lockTime = 0), - Transaction(version = 2, txIn = Nil, txOut = TxOut(Satoshi(1100_000), Script.pay2wpkh(getPublicKey(1))) :: Nil, lockTime = 0), - Transaction(version = 2, txIn = Nil, txOut = TxOut(Satoshi(1200_000), Script.pay2wpkh(getPublicKey(2))) :: Nil, lockTime = 0), + Transaction(version = 2, txIn = Nil, txOut = TxOut(Satoshi(1_000_000), Script.pay2wpkh(getPublicKey(0))) :: Nil, lockTime = 0), + Transaction(version = 2, txIn = Nil, txOut = TxOut(Satoshi(1_100_000), Script.pay2wpkh(getPublicKey(1))) :: Nil, lockTime = 0), + Transaction(version = 2, txIn = Nil, txOut = TxOut(Satoshi(1_200_000), Script.pay2wpkh(getPublicKey(2))) :: Nil, lockTime = 0), ) val bip32paths = Seq( new KeyPathWithMaster(0, new fr.acinq.bitcoin.KeyPath("m/84'/1'/0'/0/0")), @@ -66,13 +65,13 @@ class LocalOnchainKeyManagerSpec extends AnyFunSuite { { // sign all inputs and outputs - val Success(psbt1) = onchainKeyManager.signPsbt(psbt, Seq(0, 1, 2), Seq(0)) + val Success(psbt1) = onChainKeyManager.sign(psbt, Seq(0, 1, 2), Seq(0)) val signedTx = psbt1.extract().getRight Transaction.correctlySpends(signedTx, utxos, ScriptFlags.STANDARD_SCRIPT_VERIFY_FLAGS) } { // sign the first 2 inputs only - val Success(psbt1) = onchainKeyManager.signPsbt(psbt, Seq(0, 1), Seq(0)) + val Success(psbt1) = onChainKeyManager.sign(psbt, Seq(0, 1), Seq(0)) // extracting the final tx fails because no all inputs as signed assert(psbt1.extract().isLeft) assert(psbt1.getInput(2).getScriptWitness == null) @@ -80,19 +79,19 @@ class LocalOnchainKeyManagerSpec extends AnyFunSuite { { // provide a wrong derivation path for the first input val updated = psbt.updateWitnessInput(OutPoint(utxos(0), 0), utxos(0).txOut(0), null, Script.pay2pkh(getPublicKey(0)).map(scala2kmp).asJava, null, java.util.Map.of(getPublicKey(0), bip32paths(2))).getRight // wrong bip32 path - val Failure(error) = onchainKeyManager.signPsbt(updated, Seq(0, 1, 2), Seq(0)) - assert(error.getMessage.contains("cannot compute private key")) + val Failure(error) = onChainKeyManager.sign(updated, Seq(0, 1, 2), Seq(0)) + assert(error.getMessage.contains("derived public key doesn't match")) } { // provide a wrong derivation path for the first output val updated = psbt.updateWitnessOutput(0, null, null, java.util.Map.of(getPublicKey(0), bip32paths(1))).getRight // wrong path - val Failure(error) = onchainKeyManager.signPsbt(updated, Seq(0, 1, 2), Seq(0)) - assert(error.getMessage.contains("cannot compute public key")) + val Failure(error) = onChainKeyManager.sign(updated, Seq(0, 1, 2), Seq(0)) + assert(error.getMessage.contains("could not verify output 0")) } { // lie about the amount being spent val updated = psbt.updateWitnessInput(OutPoint(utxos(0), 0), utxos(0).txOut(0).copy(amount = Satoshi(10)), null, Script.pay2pkh(getPublicKey(0)).map(scala2kmp).asJava, null, java.util.Map.of(getPublicKey(0), bip32paths(0))).getRight - val Failure(error) = onchainKeyManager.signPsbt(updated, Seq(0, 1, 2), Seq(0)) + val Failure(error) = onChainKeyManager.sign(updated, Seq(0, 1, 2), Seq(0)) assert(error.getMessage.contains("utxo mismatch")) } { @@ -105,15 +104,14 @@ class LocalOnchainKeyManagerSpec extends AnyFunSuite { p4 <- p3.updateNonWitnessInput(utxos(1), 0, null, null, java.util.Map.of()) p5 <- p4.updateWitnessOutput(0, null, null, java.util.Map.of(getPublicKey(0), bip32paths(0))) } yield p5 - - val Failure(error) = onchainKeyManager.signPsbt(psbt, Seq(0, 1, 2), Seq(0)) + val Failure(error) = onChainKeyManager.sign(psbt, Seq(0, 1, 2), Seq(0)) assert(error.getMessage.contains("non-witness utxo is missing")) } { // use sighash type != SIGHASH_ALL val updated = psbt.updateWitnessInput(OutPoint(utxos(0), 0), utxos(0).txOut(0), null, Script.pay2pkh(getPublicKey(0)).map(scala2kmp).asJava, SigHash.SIGHASH_SINGLE, java.util.Map.of(getPublicKey(0), bip32paths(0))).getRight - val Failure(error) = onchainKeyManager.signPsbt(updated, Seq(0, 1, 2), Seq(0)) - assert(error.getMessage.contains("input sighashtype must be SIGHASH_ALL")) + val Failure(error) = onChainKeyManager.sign(updated, Seq(0, 1, 2), Seq(0)) + assert(error.getMessage.contains("input sighash must be SIGHASH_ALL")) } } @@ -130,7 +128,7 @@ class LocalOnchainKeyManagerSpec extends AnyFunSuite { ) data.foreach(dnc => { val Array(desc, checksum) = dnc.split('#') - assert(checksum == LocalOnchainKeyManager.descriptorChecksum(desc)) + assert(checksum == LocalOnChainKeyManager.descriptorChecksum(desc)) }) } } diff --git a/eclair-core/src/test/scala/fr/acinq/eclair/integration/basic/fixtures/MinimalNodeFixture.scala b/eclair-core/src/test/scala/fr/acinq/eclair/integration/basic/fixtures/MinimalNodeFixture.scala index 6fb7853f27..3f1138d4b3 100644 --- a/eclair-core/src/test/scala/fr/acinq/eclair/integration/basic/fixtures/MinimalNodeFixture.scala +++ b/eclair-core/src/test/scala/fr/acinq/eclair/integration/basic/fixtures/MinimalNodeFixture.scala @@ -17,7 +17,7 @@ import fr.acinq.eclair.blockchain.fee.{FeeratePerKw, FeeratesPerKw} import fr.acinq.eclair.channel._ import fr.acinq.eclair.channel.fsm.Channel import fr.acinq.eclair.crypto.TransportHandler -import fr.acinq.eclair.crypto.keymanager.{LocalChannelKeyManager, LocalNodeKeyManager, LocalOnchainKeyManager} +import fr.acinq.eclair.crypto.keymanager.{LocalChannelKeyManager, LocalNodeKeyManager} import fr.acinq.eclair.io.Peer.OpenChannelResponse import fr.acinq.eclair.io.PeerConnection.ConnectionResult import fr.acinq.eclair.io.{Peer, PeerConnection, PendingChannelsRateLimiter, Switchboard} @@ -66,15 +66,16 @@ object MinimalNodeFixture extends Assertions with Eventually with IntegrationPat def nodeParamsFor(alias: String, seed: ByteVector32): NodeParams = { NodeParams.makeNodeParams( - config = ConfigFactory.load().getConfig("eclair"), - instanceId = UUID.randomUUID(), - nodeKeyManager = new LocalNodeKeyManager(seed, Block.RegtestGenesisBlock.hash), - channelKeyManager = new LocalChannelKeyManager(seed, Block.RegtestGenesisBlock.hash), - torAddress_opt = None, - database = TestDatabases.inMemoryDb(), - blockHeight = new AtomicLong(400_000), - feerates = new AtomicReference(FeeratesPerKw.single(FeeratePerKw(253 sat))) - ).modify(_.alias).setTo(alias) + config = ConfigFactory.load().getConfig("eclair"), + instanceId = UUID.randomUUID(), + nodeKeyManager = new LocalNodeKeyManager(seed, Block.RegtestGenesisBlock.hash), + channelKeyManager = new LocalChannelKeyManager(seed, Block.RegtestGenesisBlock.hash), + onChainKeyManager_opt = None, + torAddress_opt = None, + database = TestDatabases.inMemoryDb(), + blockHeight = new AtomicLong(400_000), + feerates = new AtomicReference(FeeratesPerKw.single(FeeratePerKw(253 sat))) + ).modify(_.alias).setTo(alias) .modify(_.chainHash).setTo(Block.RegtestGenesisBlock.hash) .modify(_.routerConf.routerBroadcastInterval).setTo(1 second) .modify(_.peerConnectionConf.maxRebroadcastDelay).setTo(1 second) diff --git a/eclair-node/src/main/scala/fr/acinq/eclair/api/handlers/OnChain.scala b/eclair-node/src/main/scala/fr/acinq/eclair/api/handlers/OnChain.scala index e89b94c7ce..13ee82cbee 100644 --- a/eclair-node/src/main/scala/fr/acinq/eclair/api/handlers/OnChain.scala +++ b/eclair-node/src/main/scala/fr/acinq/eclair/api/handlers/OnChain.scala @@ -73,15 +73,14 @@ trait OnChain { val getmasterxpub: Route = postRequest("getmasterxpub") { implicit t => formFields("account".as[Long].?) { account_opt => - val xpub = this.eclairApi.getOnchainMasterPubKey(account_opt.getOrElse(0L)) + val xpub = eclairApi.getOnChainMasterPubKey(account_opt.getOrElse(0L)) complete(new JObject(List("xpub" -> JString(xpub)))) } } val getdescriptors: Route = postRequest("getdescriptors") { implicit t => - formFields("account".as[Long].?) { - (account_opt) => - val descriptors = this.eclairApi.getDescriptors(account_opt.getOrElse(0L)) + formFields("account".as[Long].?) { account_opt => + val descriptors = eclairApi.getDescriptors(account_opt.getOrElse(0L)) complete(descriptors.descriptors) } } From 0f47b52ebcfdb1b8e232d203783e3455ac90d82a Mon Sep 17 00:00:00 2001 From: sstone Date: Wed, 20 Sep 2023 13:28:37 +0200 Subject: [PATCH 5/8] Address review comments Rename `wallet` field to `walletName` Improve comments related to signature workflow and verification of wallet inputs/outputs Remove `descriptorChecksum` method: it's provided by bitcoin-kmp and does not need to be exposed --- .../main/scala/fr/acinq/eclair/Setup.scala | 9 ++++---- .../eclair/blockchain/OnChainWallet.scala | 8 ++++++- .../bitcoind/rpc/BitcoinCoreClient.scala | 16 +++++++++++++- .../channel/fund/InteractiveTxBuilder.scala | 3 ++- .../keymanager/LocalOnChainKeyManager.scala | 21 ++++++++++++------- .../crypto/keymanager/OnChainKeyManager.scala | 2 +- .../LocalOnChainKeyManagerSpec.scala | 17 --------------- 7 files changed, 42 insertions(+), 34 deletions(-) diff --git a/eclair-core/src/main/scala/fr/acinq/eclair/Setup.scala b/eclair-core/src/main/scala/fr/acinq/eclair/Setup.scala index f1653bf615..fcf3548d32 100644 --- a/eclair-core/src/main/scala/fr/acinq/eclair/Setup.scala +++ b/eclair-core/src/main/scala/fr/acinq/eclair/Setup.scala @@ -150,10 +150,10 @@ class Setup(val datadir: File, case JArray(values) => values.map(value => value.extract[String]) } eclairBackedWalletOk <- onChainKeyManager_opt match { - case Some(keyManager) if !wallets.contains(keyManager.wallet) => keyManager.createWallet(bitcoinClient) + case Some(keyManager) if !wallets.contains(keyManager.walletName) => keyManager.createWallet(bitcoinClient) case _ => Future.successful(true) } - _ = assert(eclairBackedWalletOk || onChainKeyManager_opt.map(_.wallet) != wallet, s"cannot create eclair-backed wallet=${onChainKeyManager_opt.map(_.wallet)}, check logs for details") + _ = assert(eclairBackedWalletOk || onChainKeyManager_opt.map(_.walletName) != wallet, s"cannot create eclair-backed wallet=${onChainKeyManager_opt.map(_.walletName)}, check logs for details") progress = (json \ "verificationprogress").extract[Double] ibd = (json \ "initialblockdownload").extract[Boolean] blocks = (json \ "blocks").extract[Long] @@ -252,7 +252,7 @@ class Setup(val datadir: File, finalPubkey = new AtomicReference[PublicKey](null) pubkeyRefreshDelay = FiniteDuration(config.getDuration("bitcoind.final-pubkey-refresh-delay").getSeconds, TimeUnit.SECONDS) - bitcoinClient = new BitcoinCoreClient(bitcoin, if (bitcoin.wallet == onChainKeyManager_opt.map(_.wallet)) onChainKeyManager_opt else None) with OnchainPubkeyCache { + bitcoinClient = new BitcoinCoreClient(bitcoin, if (bitcoin.wallet == onChainKeyManager_opt.map(_.walletName)) onChainKeyManager_opt else None) with OnchainPubkeyCache { val refresher: typed.ActorRef[OnchainPubkeyRefresher.Command] = system.spawn(Behaviors.supervise(OnchainPubkeyRefresher(this, finalPubkey, pubkeyRefreshDelay)).onFailure(typed.SupervisorStrategy.restart), name = "onchain-address-manager") override def getP2wpkhPubkey(renew: Boolean): PublicKey = { @@ -314,8 +314,7 @@ class Setup(val datadir: File, routerTimeout = after(FiniteDuration(config.getDuration("router.init-timeout").getSeconds, TimeUnit.SECONDS), using = system.scheduler)(Future.failed(new RuntimeException("Router initialization timed out"))) _ <- Future.firstCompletedOf(routerInitialized.future :: routerTimeout :: Nil) - _ = bitcoinClient.getReceiveAddress().map(address => logger.info(s"initial address=$address for bitcoin wallet=${bitcoinClient.rpcClient.wallet.getOrElse("")}")) - + _ = bitcoinClient.getReceiveAddress().map(address => logger.info(s"initial address=$address for bitcoin wallet=${bitcoinClient.rpcClient.wallet.getOrElse("(default)")}")) channelsListener = system.spawn(ChannelsListener(channelsListenerReady), name = "channels-listener") _ <- channelsListenerReady.future diff --git a/eclair-core/src/main/scala/fr/acinq/eclair/blockchain/OnChainWallet.scala b/eclair-core/src/main/scala/fr/acinq/eclair/blockchain/OnChainWallet.scala index 0138f857af..bc2ac328b5 100644 --- a/eclair-core/src/main/scala/fr/acinq/eclair/blockchain/OnChainWallet.scala +++ b/eclair-core/src/main/scala/fr/acinq/eclair/blockchain/OnChainWallet.scala @@ -39,7 +39,13 @@ trait OnChainChannelFunder { */ def fundTransaction(tx: Transaction, feeRate: FeeratePerKw, replaceable: Boolean, externalInputsWeight: Map[OutPoint, Long] = Map.empty)(implicit ec: ExecutionContext): Future[FundTransactionResponse] - /** Sign a PSBT. Result may be partially signed: only inputs known to our bitcoin wallet will be signed. */ + /** + * Sign a PSBT. Result may be partially signed: only inputs known to our bitcoin wallet will be signed. * + * + * @param psbt PSBT to sign + * @param ourInputs our wallet inputs. If Eclair is managing Bitcoin Core wallet keys, only these inputs will be signed. + * @param ourOutputs our wallet outputs. If Eclair is managing Bitcoin Core wallet keys, it will check that it can actually spend them (i.e re-compute private keys for them) + */ def signPsbt(psbt: Psbt, ourInputs: Seq[Int], ourOutputs: Seq[Int])(implicit ec: ExecutionContext): Future[ProcessPsbtResponse] /** diff --git a/eclair-core/src/main/scala/fr/acinq/eclair/blockchain/bitcoind/rpc/BitcoinCoreClient.scala b/eclair-core/src/main/scala/fr/acinq/eclair/blockchain/bitcoind/rpc/BitcoinCoreClient.scala index 7ae6e641c2..f8315e960b 100644 --- a/eclair-core/src/main/scala/fr/acinq/eclair/blockchain/bitcoind/rpc/BitcoinCoreClient.scala +++ b/eclair-core/src/main/scala/fr/acinq/eclair/blockchain/bitcoind/rpc/BitcoinCoreClient.scala @@ -57,7 +57,7 @@ class BitcoinCoreClient(val rpcClient: BitcoinJsonRPCClient, val onChainKeyManag implicit val formats: Formats = org.json4s.DefaultFormats onChainKeyManager_opt.foreach { keyManager => - require(rpcClient.wallet.contains(keyManager.wallet), s"eclair-backed bitcoin wallet mismatch: eclair-signer.conf uses wallet=${keyManager.wallet}, but eclair.conf uses wallet=${rpcClient.wallet.getOrElse("")}") + require(rpcClient.wallet.contains(keyManager.walletName), s"eclair-backed bitcoin wallet mismatch: eclair-signer.conf uses wallet=${keyManager.walletName}, but eclair.conf uses wallet=${rpcClient.wallet.getOrElse("")}") } val useEclairSigner = onChainKeyManager_opt.nonEmpty @@ -753,7 +753,21 @@ object BitcoinCoreClient { def toSatoshi(btcAmount: BigDecimal): Satoshi = Satoshi(btcAmount.bigDecimal.scaleByPowerOfTen(8).longValue) + /** + * Bitcoin Core descriptor, as used in Bitcoin Core's RPC API. + * + * @param desc Descriptor string representation + * @param internal True if this descriptor is used to generate change addresses. False if this descriptor is used to generate receiving addresses; defined only for active descriptors + * @param timestamp The creation time of the descriptor + * @param active Whether this descriptor is currently used to generate new addresses + */ case class Descriptor(desc: String, internal: Boolean = false, timestamp: Long, active: Boolean = true) + /** + * Descriptors for a specific Bitcoin wallet. + * + * @param wallet_name wallet name + * @param descriptors list of wallet descriptors + */ case class Descriptors(wallet_name: String, descriptors: Seq[Descriptor]) } \ No newline at end of file diff --git a/eclair-core/src/main/scala/fr/acinq/eclair/channel/fund/InteractiveTxBuilder.scala b/eclair-core/src/main/scala/fr/acinq/eclair/channel/fund/InteractiveTxBuilder.scala index 998f8e5d11..dc358fbbda 100644 --- a/eclair-core/src/main/scala/fr/acinq/eclair/channel/fund/InteractiveTxBuilder.scala +++ b/eclair-core/src/main/scala/fr/acinq/eclair/channel/fund/InteractiveTxBuilder.scala @@ -773,7 +773,6 @@ private class InteractiveTxBuilder(replyTo: ActorRef[InteractiveTxBuilder.Respon if (unsignedTx.localInputs.isEmpty) { context.self ! SignTransactionResult(PartiallySignedSharedTransaction(unsignedTx, TxSignatures(fundingParams.channelId, tx, Nil, sharedSig_opt))) } else { - // We only sign our wallet inputs, and check that we can spend our wallet outputs. val ourWalletInputs = unsignedTx.localInputs.map(i => tx.txIn.indexWhere(_.outPoint == i.outPoint)) val ourWalletOutputs = unsignedTx.localOutputs.flatMap { case Output.Local.Change(_, amount, pubkeyScript) => Some(tx.txOut.indexWhere(output => output.amount == amount && output.publicKeyScript == pubkeyScript)) @@ -782,6 +781,8 @@ private class InteractiveTxBuilder(replyTo: ActorRef[InteractiveTxBuilder.Respon // We trust that non-change outputs are valid: this only works if the entry point for creating such outputs is trusted (for example, a secure API call). case _: Output.Local.NonChange => None } + // We track our wallet inputs and outputs, so we can verify them when we sign the transaction: if Eclair is managing bitcoin core wallet keys, it will + // only sign our wallet inputs, and check that it can re-compute private keys for our wallet outputs context.pipeToSelf(wallet.signPsbt(new Psbt(tx), ourWalletInputs, ourWalletOutputs).map { response => val localOutpoints = unsignedTx.localInputs.map(_.outPoint).toSet diff --git a/eclair-core/src/main/scala/fr/acinq/eclair/crypto/keymanager/LocalOnChainKeyManager.scala b/eclair-core/src/main/scala/fr/acinq/eclair/crypto/keymanager/LocalOnChainKeyManager.scala index 35cdeeed07..56cf4d86c0 100644 --- a/eclair-core/src/main/scala/fr/acinq/eclair/crypto/keymanager/LocalOnChainKeyManager.scala +++ b/eclair-core/src/main/scala/fr/acinq/eclair/crypto/keymanager/LocalOnChainKeyManager.scala @@ -34,7 +34,6 @@ import scala.jdk.CollectionConverters.{CollectionHasAsScala, MapHasAsScala} import scala.util.{Failure, Success, Try} object LocalOnChainKeyManager extends Logging { - def descriptorChecksum(span: String): String = fr.acinq.bitcoin.Descriptor.checksum(span) /** * Load a configuration file and create an on-chain key manager @@ -62,7 +61,13 @@ object LocalOnChainKeyManager extends Logging { } } -class LocalOnChainKeyManager(override val wallet: String, seed: ByteVector, override val walletTimestamp: TimestampSecond, chainHash: ByteVector32) extends OnChainKeyManager with Logging { +/** + * A manager for on-chain keys used by Eclair, to be used in combination with a watch-only descriptor-based wallet managed by Bitcoin Core. + * In this setup, Bitcoin Core handles all non-sensitive wallet tasks (including watching the blockchain and building transactions), while + * Eclair is in charge of signing transactions. + * This is an advanced feature particularly suited when Eclair runs in a secure runtime. + */ +class LocalOnChainKeyManager(override val walletName: String, seed: ByteVector, override val walletTimestamp: TimestampSecond, chainHash: ByteVector32) extends OnChainKeyManager with Logging { import LocalOnChainKeyManager._ @@ -100,11 +105,11 @@ class LocalOnChainKeyManager(override val wallet: String, seed: ByteVector, over override def createWallet(rpcClient: BitcoinJsonRPCClient)(implicit ec: ExecutionContext): Future[Boolean] = { if (walletTimestamp < (TimestampSecond.now() - 2.hours)) { - logger.warn(s"eclair-backed wallet descriptors for wallet=$wallet are too old to be automatically imported into bitcoin core, you will need to manually import them and select how far back to rescan") + logger.warn(s"eclair-backed wallet descriptors for wallet=$walletName are too old to be automatically imported into bitcoin core, you will need to manually import them and select how far back to rescan") Future.successful(false) } else { - logger.info(s"creating a new on-chain eclair-backed wallet in bitcoind: $wallet") - rpcClient.invoke("createwallet", wallet, /* disable_private_keys */ true, /* blank */ true, /* passphrase */ "", /* avoid_reuse */ false, /* descriptors */ true, /* load_on_startup */ true).flatMap(_ => { + logger.info(s"creating a new on-chain eclair-backed wallet in bitcoind: $walletName") + rpcClient.invoke("createwallet", walletName, /* disable_private_keys */ true, /* blank */ true, /* passphrase */ "", /* avoid_reuse */ false, /* descriptors */ true, /* load_on_startup */ true).flatMap(_ => { logger.info(s"importing new descriptors ${descriptors(0).descriptors}") rpcClient.invoke("importdescriptors", descriptors(0).descriptors).collect { case JArray(results) => results.forall(item => { @@ -132,9 +137,9 @@ class LocalOnChainKeyManager(override val wallet: String, seed: ByteVector, over // 84'/{0'/1'}/0'/1/* for change addresses val receiveDesc = s"wpkh([$fingerPrintHex/$keyPath]${encode(accountPub, prefix)}/0/*)" val changeDesc = s"wpkh([$fingerPrintHex/$keyPath]${encode(accountPub, prefix)}/1/*)" - Descriptors(wallet_name = wallet, descriptors = List( - Descriptor(desc = s"$receiveDesc#${descriptorChecksum(receiveDesc)}", internal = false, active = true, timestamp = walletTimestamp.toLong), - Descriptor(desc = s"$changeDesc#${descriptorChecksum(changeDesc)}", internal = true, active = true, timestamp = walletTimestamp.toLong), + Descriptors(wallet_name = walletName, descriptors = List( + Descriptor(desc = s"$receiveDesc#${fr.acinq.bitcoin.Descriptor.checksum(receiveDesc)}", internal = false, active = true, timestamp = walletTimestamp.toLong), + Descriptor(desc = s"$changeDesc#${fr.acinq.bitcoin.Descriptor.checksum(changeDesc)}", internal = true, active = true, timestamp = walletTimestamp.toLong), )) } diff --git a/eclair-core/src/main/scala/fr/acinq/eclair/crypto/keymanager/OnChainKeyManager.scala b/eclair-core/src/main/scala/fr/acinq/eclair/crypto/keymanager/OnChainKeyManager.scala index 867289b4bd..969253a803 100644 --- a/eclair-core/src/main/scala/fr/acinq/eclair/crypto/keymanager/OnChainKeyManager.scala +++ b/eclair-core/src/main/scala/fr/acinq/eclair/crypto/keymanager/OnChainKeyManager.scala @@ -27,7 +27,7 @@ import scala.concurrent.{ExecutionContext, Future} import scala.util.Try trait OnChainKeyManager { - def wallet: String + def walletName: String /** * @return the creation time of the wallet managed by this key manager diff --git a/eclair-core/src/test/scala/fr/acinq/eclair/crypto/keymanager/LocalOnChainKeyManagerSpec.scala b/eclair-core/src/test/scala/fr/acinq/eclair/crypto/keymanager/LocalOnChainKeyManagerSpec.scala index 353d34c3da..b550e452ea 100644 --- a/eclair-core/src/test/scala/fr/acinq/eclair/crypto/keymanager/LocalOnChainKeyManagerSpec.scala +++ b/eclair-core/src/test/scala/fr/acinq/eclair/crypto/keymanager/LocalOnChainKeyManagerSpec.scala @@ -114,21 +114,4 @@ class LocalOnChainKeyManagerSpec extends AnyFunSuite { assert(error.getMessage.contains("input sighash must be SIGHASH_ALL")) } } - - test("compute descriptor checksums") { - val data = Seq( - "pkh([6ded4eb8/44h/0h/0h]xpub6C6N5WVF5zmurBR52MZZj8Jxm6eDiKyM4wFCm7xTYBEsAvJPqBKp2u2K7RTsZaYDN8duBWq4acrD4vrwjaKHTYuntGjL334nVHtLNuaj5Mu/0/*)#5mzpq0w6", - "wpkh([6ded4eb8/84h/0h/0h]xpub6CDeom4xT3Wg7BuyXU2Sd9XerTKttyfxRwJE36mi5HxFYpYdtdwM76Zx8swPnc6zxuArMYJgjNy91fJ13YtGPHgf49YqA8KdXg6D69tzNFh/0/*)#refya6f0", - "sh(wpkh([6ded4eb8/49h/0h/0h]xpub6Cb8jR9kYsfC6kj9CsE18SyudWjW2V3FnBFkT2oqq6n7NWWvJrjhFin3sAYg8X7ApX8iPophBa98mo4nMvSxnqrXvpnwaRopecQz859Ai1s/0/*))#xrhyhtvl", - "tr([6ded4eb8/86h/0h/0h]xpub6CDp1iw76taes3pkqfiJ6PYhwURkaYksJ62CrrdTVr6ow9wR9mKAtUGoZQqb8pRDiq2F8k31tYrrJjVGTRSLYGQ7nYpmewH94ThsAgDxJ4h/0/*)#2nm7drky", - "pkh([6ded4eb8/44h/0h/0h]xpub6C6N5WVF5zmurBR52MZZj8Jxm6eDiKyM4wFCm7xTYBEsAvJPqBKp2u2K7RTsZaYDN8duBWq4acrD4vrwjaKHTYuntGjL334nVHtLNuaj5Mu/1/*)#908qa67z", - "wpkh([6ded4eb8/84h/0h/0h]xpub6CDeom4xT3Wg7BuyXU2Sd9XerTKttyfxRwJE36mi5HxFYpYdtdwM76Zx8swPnc6zxuArMYJgjNy91fJ13YtGPHgf49YqA8KdXg6D69tzNFh/1/*)#jdv9q0eh", - "sh(wpkh([6ded4eb8/49h/0h/0h]xpub6Cb8jR9kYsfC6kj9CsE18SyudWjW2V3FnBFkT2oqq6n7NWWvJrjhFin3sAYg8X7ApX8iPophBa98mo4nMvSxnqrXvpnwaRopecQz859Ai1s/1/*))#nzej05eq", - "tr([6ded4eb8/86h/0h/0h]xpub6CDp1iw76taes3pkqfiJ6PYhwURkaYksJ62CrrdTVr6ow9wR9mKAtUGoZQqb8pRDiq2F8k31tYrrJjVGTRSLYGQ7nYpmewH94ThsAgDxJ4h/1/*)#m87lskxu" - ) - data.foreach(dnc => { - val Array(desc, checksum) = dnc.split('#') - assert(checksum == LocalOnChainKeyManager.descriptorChecksum(desc)) - }) - } } From 7cd8b148ff3a10447f5d353e85f20edbfbfaa90f Mon Sep 17 00:00:00 2001 From: Bastien Teinturier <31281497+t-bast@users.noreply.github.com> Date: Wed, 20 Sep 2023 17:04:47 +0200 Subject: [PATCH 6/8] Simplify replaceable tx funding (#2745) * Refactor `dummySignedCommitTx` We only need the weight of the signed commit tx, it was error-prone to provide what looks like a signed commit tx. * Simplify replaceable tx funding We were previously signing twice (with makes a call to `bitcoind`), just to get the final weights and adjust the change outputs. This was unnecessary, as we can adjust the weights before adding inputs. We were also duplicating the checks where we verify that `bitcoind` is malicious. We only need to check that once, during the final signing step. --- .../channel/publish/ReplaceableTxFunder.scala | 79 ++++++------------- 1 file changed, 24 insertions(+), 55 deletions(-) diff --git a/eclair-core/src/main/scala/fr/acinq/eclair/channel/publish/ReplaceableTxFunder.scala b/eclair-core/src/main/scala/fr/acinq/eclair/channel/publish/ReplaceableTxFunder.scala index d5f8957e35..a5304cf4bd 100644 --- a/eclair-core/src/main/scala/fr/acinq/eclair/channel/publish/ReplaceableTxFunder.scala +++ b/eclair-core/src/main/scala/fr/acinq/eclair/channel/publish/ReplaceableTxFunder.scala @@ -22,7 +22,6 @@ import akka.actor.typed.{ActorRef, Behavior} import fr.acinq.bitcoin.psbt.Psbt import fr.acinq.bitcoin.scalacompat.{ByteVector32, OutPoint, Satoshi, Script, Transaction, TxOut} import fr.acinq.eclair.NotificationsLogger.NotifyNodeOperator -import fr.acinq.eclair.blockchain.OnChainWallet import fr.acinq.eclair.blockchain.bitcoind.rpc.BitcoinCoreClient import fr.acinq.eclair.blockchain.bitcoind.rpc.BitcoinCoreClient.{FundTransactionOptions, InputWeight} import fr.acinq.eclair.blockchain.fee.FeeratePerKw @@ -87,9 +86,10 @@ object ReplaceableTxFunder { } } - private def dummySignedCommitTx(commitment: FullCommitment): CommitTx = { + private def commitWeight(commitment: FullCommitment): Int = { val unsignedCommitTx = commitment.localCommit.commitTxAndRemoteSig.commitTx - addSigs(unsignedCommitTx, PlaceHolderPubKey, PlaceHolderPubKey, PlaceHolderSig, PlaceHolderSig) + val dummySignedCommitTx = addSigs(unsignedCommitTx, PlaceHolderPubKey, PlaceHolderPubKey, PlaceHolderSig, PlaceHolderSig) + dummySignedCommitTx.tx.weight() } /** @@ -116,7 +116,7 @@ object ReplaceableTxFunder { case _: ClaimHtlcSuccessTx => Transactions.claimHtlcSuccessWeight case _: LegacyClaimHtlcSuccessTx => Transactions.claimHtlcSuccessWeight case _: ClaimHtlcTimeoutTx => Transactions.claimHtlcTimeoutWeight - case _: ClaimLocalAnchorOutputTx => dummySignedCommitTx(commitment).tx.weight() + Transactions.claimAnchorOutputMinWeight + case _: ClaimLocalAnchorOutputTx => commitWeight(commitment) + Transactions.claimAnchorOutputMinWeight } Transactions.fee2rate(maxFee, weight) } @@ -164,9 +164,9 @@ object ReplaceableTxFunder { val dustLimit = commitment.localParams.dustLimit val targetFee = previousTx.signedTxWithWitnessData match { case _: ClaimLocalAnchorWithWitnessData => - val commitTx = dummySignedCommitTx(commitment) - val totalWeight = previousTx.signedTx.weight() + commitTx.tx.weight() - weight2fee(targetFeerate, totalWeight) - commitTx.fee + val commitFee = commitment.localCommit.commitTxAndRemoteSig.commitTx.fee + val totalWeight = previousTx.signedTx.weight() + commitWeight(commitment) + weight2fee(targetFeerate, totalWeight) - commitFee case _ => weight2fee(targetFeerate, previousTx.signedTx.weight()) } @@ -378,7 +378,7 @@ private class ReplaceableTxFunder(nodeParams: NodeParams, case Right(signedTx) => val actualFees = kmp2scala(processPsbtResponse.psbt.computeFees()) val actualWeight = locallySignedTx match { - case _: ClaimLocalAnchorWithWitnessData => signedTx.weight() + dummySignedCommitTx(cmd.commitment).tx.weight() + case _: ClaimLocalAnchorWithWitnessData => signedTx.weight() + commitWeight(cmd.commitment) case _ => signedTx.weight() } val actualFeerate = Transactions.fee2rate(actualFees, actualWeight) @@ -425,80 +425,49 @@ private class ReplaceableTxFunder(nodeParams: NodeParams, } private def addInputs(anchorTx: ClaimLocalAnchorWithWitnessData, targetFeerate: FeeratePerKw, commitment: FullCommitment): Future[(ClaimLocalAnchorWithWitnessData, Satoshi)] = { - import fr.acinq.bitcoin.scalacompat.KotlinUtils._ + import fr.acinq.bitcoin.scalacompat.KotlinUtils val dustLimit = commitment.localParams.dustLimit - val commitTx = dummySignedCommitTx(commitment).tx + val commitTxWeight = commitWeight(commitment) // NB: fundrawtransaction requires at least one output, and may add at most one additional change output. // Since the purpose of this transaction is just to do a CPFP, the resulting tx should have a single change output // (note that bitcoind doesn't let us publish a transaction with no outputs). To work around these limitations, we // start with a dummy output and later merge that dummy output with the optional change output added by bitcoind. - val txNotFunded = anchorTx.txInfo.tx.copy(txOut = TxOut(dustLimit, Script.pay2wpkh(PlaceHolderPubKey)) :: Nil) + val dummyChangeOutput = TxOut(dustLimit, Script.pay2wpkh(PlaceHolderPubKey)) + val txNotFunded = anchorTx.txInfo.tx.copy(txOut = dummyChangeOutput :: Nil) // The anchor transaction is paying for the weight of the commitment transaction. - val anchorWeight = Seq(InputWeight(anchorTx.txInfo.input.outPoint, anchorInputWeight + commitTx.weight())) - - def makeSingleOutputTx(fundTxResponse: OnChainWallet.FundTransactionResponse): Future[Transaction] = { + // We remove the weight of the artificially added change output, because we will remove that output after funding. + val anchorWeight = Seq(InputWeight(anchorTx.txInfo.input.outPoint, anchorInputWeight + commitTxWeight - KotlinUtils.scala2kmp(dummyChangeOutput).weight())) + bitcoinClient.fundTransaction(txNotFunded, FundTransactionOptions(targetFeerate, inputWeights = anchorWeight)).flatMap { fundTxResponse => // Bitcoin Core may not preserve the order of inputs, we need to make sure the anchor is the first input. val txIn = anchorTx.txInfo.tx.txIn ++ fundTxResponse.tx.txIn.filterNot(_.outPoint == anchorTx.txInfo.input.outPoint) + // The commitment transaction was already paying some fees that we're paying again in the anchor transaction since + // we included the commit weight, so we need to increase our change output to avoid overshooting the feerate. + val commitFee = commitment.localCommit.commitTxAndRemoteSig.commitTx.fee fundTxResponse.changePosition match { case Some(changePos) => - val changeOutput = fundTxResponse.tx.txOut(changePos).copy(amount = fundTxResponse.tx.txOut.map(_.amount).sum) + val changeOutput = fundTxResponse.tx.txOut(changePos).copy(amount = fundTxResponse.tx.txOut.map(_.amount).sum + commitFee) val txSingleOutput = fundTxResponse.tx.copy(txIn = txIn, txOut = Seq(changeOutput)) - Future.successful(txSingleOutput) + Future.successful(anchorTx.updateTx(txSingleOutput), fundTxResponse.amountIn) case None => bitcoinClient.getP2wpkhPubkeyHashForChange().map(pubkeyHash => { // replace PlaceHolderPubKey with a real wallet key - fundTxResponse.tx.copy(txIn = txIn, txOut = Seq(TxOut(dustLimit, Script.pay2wpkh(pubkeyHash)))) + val txSingleOutput = fundTxResponse.tx.copy(txIn = txIn, txOut = Seq(TxOut(dustLimit + commitFee, Script.pay2wpkh(pubkeyHash)))) + (anchorTx.updateTx(txSingleOutput), fundTxResponse.amountIn) }) } } - - for { - fundTxResponse <- bitcoinClient.fundTransaction(txNotFunded, FundTransactionOptions(targetFeerate, inputWeights = anchorWeight)) - txSingleOutput <- makeSingleOutputTx(fundTxResponse) - changeOutput = txSingleOutput.txOut.head - ourWalletInputs = txSingleOutput.txIn.indices.tail // all inputs except the first one - ourWalletOutputs = Seq(0) // one change output - // We ask bitcoind to sign the wallet inputs to learn their final weight and adjust the change amount. - processPsbtResponse <- bitcoinClient.signPsbt(new Psbt(txSingleOutput), ourWalletInputs, ourWalletOutputs) - // we cannot extract the final tx from the psbt because it is not fully signed yet - partiallySignedTx = processPsbtResponse.partiallySignedTx - dummySignedTx = addSigs(anchorTx.updateTx(partiallySignedTx).txInfo, PlaceHolderSig) - packageWeight = commitTx.weight() + dummySignedTx.tx.weight() - // above, we asked bitcoin core to use the package weight to estimate fees when it built and funded this transaction, so we - // use the same package weight here to compute the actual fee rate that we get - actualFeerate = Transactions.fee2rate(processPsbtResponse.psbt.computeFees(), packageWeight) - _ = require(actualFeerate < targetFeerate * 2, s"actual fee rate $actualFeerate is more than twice the requested fee rate $targetFeerate") - - anchorTxFee = weight2fee(targetFeerate, packageWeight) - weight2fee(commitment.localCommit.spec.commitTxFeerate, commitTx.weight()) - changeAmount = dustLimit.max(fundTxResponse.amountIn - anchorTxFee) - fundedTx = txSingleOutput.copy(txOut = Seq(changeOutput.copy(amount = changeAmount))) - } yield { - (anchorTx.updateTx(fundedTx), fundTxResponse.amountIn) - } } private def addInputs(htlcTx: HtlcWithWitnessData, targetFeerate: FeeratePerKw, commitment: FullCommitment): Future[(HtlcWithWitnessData, Satoshi)] = { - import fr.acinq.bitcoin.scalacompat.KotlinUtils._ - val htlcInputWeight = InputWeight(htlcTx.txInfo.input.outPoint, htlcTx.txInfo match { case _: HtlcSuccessTx => commitment.params.commitmentFormat.htlcSuccessInputWeight case _: HtlcTimeoutTx => commitment.params.commitmentFormat.htlcTimeoutInputWeight }) - bitcoinClient.fundTransaction(htlcTx.txInfo.tx, FundTransactionOptions(targetFeerate, changePosition = Some(1), inputWeights = Seq(htlcInputWeight))).flatMap(fundTxResponse => { + bitcoinClient.fundTransaction(htlcTx.txInfo.tx, FundTransactionOptions(targetFeerate, changePosition = Some(1), inputWeights = Seq(htlcInputWeight))).map(fundTxResponse => { // Bitcoin Core may not preserve the order of inputs, we need to make sure the htlc is the first input. val fundedTx = fundTxResponse.tx.copy(txIn = htlcTx.txInfo.tx.txIn ++ fundTxResponse.tx.txIn.filterNot(_.outPoint == htlcTx.txInfo.input.outPoint)) - val ourWalletInputs = fundedTx.txIn.indices.tail // all inputs except the first one - val ourWalletOutputs = if (fundedTx.txOut.size > 1) Seq(1) else Nil // there may not be a change output - val unsignedTx = htlcTx.updateTx(fundedTx) - bitcoinClient.signPsbt(new Psbt(fundedTx), ourWalletInputs, ourWalletOutputs).map(processPsbtResponse => { - val actualFees: Satoshi = processPsbtResponse.psbt.computeFees() - require(actualFees == fundTxResponse.fee, s"Bitcoin Core fees (${fundTxResponse.fee}) do not match ours ($actualFees)") - val packageWeight = fundedTx.weight() + htlcInputWeight.weight - val actualFeerate = Transactions.fee2rate(fundTxResponse.fee, packageWeight.toInt) - require(actualFeerate < targetFeerate * 2, s"actual fee rate $actualFeerate is more than twice the requested fee rate $targetFeerate") - (unsignedTx, fundTxResponse.amountIn) - }) + (htlcTx.updateTx(fundedTx), fundTxResponse.amountIn) }) } } From bf6f240076a40ffb2be9559d350ab64c9feb94b4 Mon Sep 17 00:00:00 2001 From: Fabrice Drouin Date: Thu, 21 Sep 2023 13:48:20 +0200 Subject: [PATCH 7/8] Remove code to automatically create eclair-managed wallet (#2746) Remove code to automatically create eclair-managed wallet This code made the onchain key manager more complex, and for older wallets (when nodes are moved to a new machine for example) we need to provide a manual process for creating a new empty wallet and importing descriptors generated by Eclair. => It is simpler to always use this manual process. --- docs/Guides.md | 2 +- ...CoreKeys.md => ManagingBitcoinCoreKeys.md} | 92 +++++-------------- .../main/scala/fr/acinq/eclair/Setup.scala | 5 - .../keymanager/LocalOnChainKeyManager.scala | 23 +---- .../crypto/keymanager/OnChainKeyManager.scala | 11 --- .../bitcoind/BitcoinCoreClientSpec.scala | 6 +- .../blockchain/bitcoind/BitcoindService.scala | 16 +++- .../publish/ReplaceableTxPublisherSpec.scala | 3 +- 8 files changed, 43 insertions(+), 115 deletions(-) rename docs/{BitcoinCoreKeys.md => ManagingBitcoinCoreKeys.md} (52%) diff --git a/docs/Guides.md b/docs/Guides.md index 85c07ed300..91c3f00447 100644 --- a/docs/Guides.md +++ b/docs/Guides.md @@ -4,7 +4,7 @@ This section contains how-to guides for more advanced scenarios: * [Customize Logging](./Logging.md) * [Customize Features](./Features.md) -* [Manage Bitcoin Core's private keys](./BitcoinCoreKeys.md) +* [Manage Bitcoin Core's private keys](./ManagingBitcoinCoreKeys.md) * [Use Tor with Eclair](./Tor.md) * [Multipart Payments](./MultipartPayments.md) * [Trampoline Payments](./TrampolinePayments.md) diff --git a/docs/BitcoinCoreKeys.md b/docs/ManagingBitcoinCoreKeys.md similarity index 52% rename from docs/BitcoinCoreKeys.md rename to docs/ManagingBitcoinCoreKeys.md index ba8b9de14f..66f0e8f046 100644 --- a/docs/BitcoinCoreKeys.md +++ b/docs/ManagingBitcoinCoreKeys.md @@ -16,12 +16,12 @@ You can use any BIP39-compatible tool, including most hardware wallets. A signer configuration file uses the HOCON format that we already use for `eclair.conf` and must include the following options: - key | description ---------------------------|-------------------------------------------------------------------------- - eclair.signer.wallet | wallet name - eclair.signer.mnemonics | BIP39 mnemonic words - eclair.signer.passphrase | passphrase - eclair.signer.timestamp | wallet creation UNIX timestamp. Set to the current time for new wallets. + key | description +--------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ + eclair.signer.wallet | wallet name + eclair.signer.mnemonics | BIP39 mnemonic words + eclair.signer.passphrase | passphrase + eclair.signer.timestamp | wallet creation UNIX timestamp. Bitcoin core will rescan the blockchain from this UNIX timestamp. Set it to the wallet creation timestamp for simplicity, or a later date if you only have recent UTXOs and you know what you are doing. This is an example of `eclair-signer.conf` configuration file: @@ -38,38 +38,11 @@ This is an example of `eclair-signer.conf` configuration file: } ``` -### 3. Configure Eclair to handle private keys for this wallet +### 3. Use Eclair to generate descriptors and import them into a new bitcoin wallet -Set `eclair.bitcoind.wallet` to the name of the wallet in your `eclair-signer.conf` file (`eclair` in the example above) and restart Eclair. -Eclair will automatically create a new, empty, descriptor-enabled, watch-only wallet in Bitcoin Core and import its descriptors. +Restart eclair, without changing `eclair.bitcoind.wallet` (so it uses the default wallet or the previously used bitcoin wallet for existing nodes). -:warning: Eclair will not import descriptors if the timestamp set in your `eclair-signer.conf` is more than 2 hours old. If the mnemonics and -passphrase that your are using are new, you can safely update this timestamp, but if they have been used before you will need to follow -the steps described in the next section. - -You now have a Bitcoin Core watch-only wallet for which only your Eclair node can sign transactions. This Bitcoin Core wallet can -safely be copied to another Bitcoin Core node to monitor your on-chain funds. - -You can also use `eclair-cli getmasterxpub` to get a BIP32 extended public key that you can import into any compatible Bitcoin wallet -to create a watch-only wallet (Electrum for example) that you can use to monitor your Bitcoin Core balance. - -:warning: this means that your Bitcoin Core wallet cannot send funds on its own (since it cannot access private keys to sign transactions). -To send funds on-chain you must use `eclair-cli sendonchain`. - -:warning: to backup the private keys of this wallet you must either backup your mnemonic code and passphrase, or backup the `eclair-signer.conf` file in your eclair -directory (default is `~/.eclair`) along with your channels and node seed files. - -:warning: You can also initialize a backup on-chain wallet with the same mnemonic code and passphrase (on a hardware wallet for example), but be warned that using them may interfere with your node's operations (for example you may end up -double-spending funding transactions generated by your node). - -## Importing an existing Eclair-backed Bitcoin Core wallet - -The steps above described how you can simply switch to a new Eclair-backed bitcoin wallet. -Follow the steps below if you are already using an Eclair-backed bitcoin wallet that you want to move to another Bitcoin Core node. - -### 1. Create an empty, descriptor-enabled, watch-only wallet in Bitcoin Core - -Start by creating a watch-only wallet on your new Bitcoin Core node. +Create a new empty, decriptor-enabled wallet on your new Bitcoin Core node. :warning: The name must match the one that you set in `eclair-signer.conf` (here we use "eclair") @@ -77,28 +50,6 @@ Start by creating a watch-only wallet on your new Bitcoin Core node. $ bitcoin-cli -named createwallet wallet_name=eclair disable_private_keys=true blank=true descriptors=true load_on_startup=true ``` -### 2. Import public descriptors generated by Eclair - -Calling `eclair-cli getdescriptors` on your existing Eclair node will return public wallet descriptors in a format that is compatible with Bitcoin Core. -This is an example of descriptors generated by Eclair: - -```json -[ - { - "desc": "wpkh([0d9250da/84h/1h/0h]tpubDDGF9PnrXww2h1mKNjKiXoqdDFGEcZGCZUNq7g26LdzKXKiE31RrFWsogPy1uMLrbG8ksQ8eJS6u6KFLjYUUSVJRuwmMD2SYCr8uG1TcRgM/0/*)#jz5n2pcp", - "internal": false, - "timestamp": 1686055705, - "active": true - }, - { - "desc": "wpkh([0d9250da/84h/1h/0h]tpubDDGF9PnrXww2h1mKNjKiXoqdDFGEcZGCZUNq7g26LdzKXKiE31RrFWsogPy1uMLrbG8ksQ8eJS6u6KFLjYUUSVJRuwmMD2SYCr8uG1TcRgM/1/*)#rk3jh5ge", - "internal": true, - "timestamp": 1686055705, - "active": true - } -] -``` - Generate the descriptors with your Eclair node and import them into a Bitcoin node with the following commands: ```shell @@ -106,17 +57,24 @@ $ eclair-cli getdescriptors | jq --raw-output -c > descriptors.json $ cat descriptors.json | xargs -0 bitcoin-cli -rpcwallet=eclair importdescriptors ``` -:warning: Importing descriptors can take a long time, and your Bitcoin Core node will not be usable until it's done +Bitcoin core will import descriptors and rescan the blockchain from the time set in `eclair-signer.conf`. +This can take a long time (if you're moving an old existing node to a new setup for example) and your Bitcoin Core node will not be usable until it's done. -### 3. Configure Eclair to use your new Bitcoin Core node +### 4. Configure Eclair to use the wallet you created and restart Eclair -Once your new Bitcoin Core node has finished importing the descriptors, it is ready to be used by Eclair. +In your `eclair.conf`, set `eclair.bitcoind.wallet` to the name of the wallet in `eclair-signer.conf`, and restart Eclair. -In your `eclair.conf`: +You now have a Bitcoin Core watch-only wallet for which only your Eclair node can sign transactions. This Bitcoin Core wallet can +safely be copied to another Bitcoin Core node to monitor your on-chain funds. -- set `eclair.bitcoind.host` to the address of your new Bitcoin Core node -- set `eclair.bitcoind.wallet` to the name of the wallet in `eclair-signer.conf` -- set `eclair.bitcoind.zmqblock` and `eclair.bitcoind.zmqtx` to use your new Bitcoin Core node -- update other field in the `eclair.bitcoind` section if necessary (`rpcport`, `auth`, `rpcuser`, `rpcuser`, etc) +:warning: this means that your Bitcoin Core wallet cannot send funds on its own (since it cannot access private keys to sign transactions). +To send funds on-chain you must use `eclair-cli sendonchain`. -Restart Eclair and it will start using your new Bitcoin Core node. +:warning: to backup the private keys of this wallet you must either backup your mnemonic code and passphrase, or backup the `eclair-signer.conf` file in your eclair +directory (default is `~/.eclair`) along with your channels and node seed files. + +:warning: You can also initialize a backup on-chain wallet with the same mnemonic code and passphrase (on a hardware wallet for example), but be warned that using them may interfere with your node's operations (for example you may end up +double-spending funding transactions generated by your node). + +You can also use `eclair-cli getmasterxpub` to get a BIP32 extended public key that you can import into any compatible Bitcoin wallet +to create a watch-only wallet (Electrum for example) that you can use to monitor your Bitcoin Core balance. diff --git a/eclair-core/src/main/scala/fr/acinq/eclair/Setup.scala b/eclair-core/src/main/scala/fr/acinq/eclair/Setup.scala index fcf3548d32..d52eb8860d 100644 --- a/eclair-core/src/main/scala/fr/acinq/eclair/Setup.scala +++ b/eclair-core/src/main/scala/fr/acinq/eclair/Setup.scala @@ -149,11 +149,6 @@ class Setup(val datadir: File, .collect { case JArray(values) => values.map(value => value.extract[String]) } - eclairBackedWalletOk <- onChainKeyManager_opt match { - case Some(keyManager) if !wallets.contains(keyManager.walletName) => keyManager.createWallet(bitcoinClient) - case _ => Future.successful(true) - } - _ = assert(eclairBackedWalletOk || onChainKeyManager_opt.map(_.walletName) != wallet, s"cannot create eclair-backed wallet=${onChainKeyManager_opt.map(_.walletName)}, check logs for details") progress = (json \ "verificationprogress").extract[Double] ibd = (json \ "initialblockdownload").extract[Boolean] blocks = (json \ "blocks").extract[Long] diff --git a/eclair-core/src/main/scala/fr/acinq/eclair/crypto/keymanager/LocalOnChainKeyManager.scala b/eclair-core/src/main/scala/fr/acinq/eclair/crypto/keymanager/LocalOnChainKeyManager.scala index 56cf4d86c0..38779fdd8d 100644 --- a/eclair-core/src/main/scala/fr/acinq/eclair/crypto/keymanager/LocalOnChainKeyManager.scala +++ b/eclair-core/src/main/scala/fr/acinq/eclair/crypto/keymanager/LocalOnChainKeyManager.scala @@ -67,7 +67,7 @@ object LocalOnChainKeyManager extends Logging { * Eclair is in charge of signing transactions. * This is an advanced feature particularly suited when Eclair runs in a secure runtime. */ -class LocalOnChainKeyManager(override val walletName: String, seed: ByteVector, override val walletTimestamp: TimestampSecond, chainHash: ByteVector32) extends OnChainKeyManager with Logging { +class LocalOnChainKeyManager(override val walletName: String, seed: ByteVector, val walletTimestamp: TimestampSecond, chainHash: ByteVector32) extends OnChainKeyManager with Logging { import LocalOnChainKeyManager._ @@ -103,27 +103,6 @@ class LocalOnChainKeyManager(override val walletName: String, seed: ByteVector, (pub, address) } - override def createWallet(rpcClient: BitcoinJsonRPCClient)(implicit ec: ExecutionContext): Future[Boolean] = { - if (walletTimestamp < (TimestampSecond.now() - 2.hours)) { - logger.warn(s"eclair-backed wallet descriptors for wallet=$walletName are too old to be automatically imported into bitcoin core, you will need to manually import them and select how far back to rescan") - Future.successful(false) - } else { - logger.info(s"creating a new on-chain eclair-backed wallet in bitcoind: $walletName") - rpcClient.invoke("createwallet", walletName, /* disable_private_keys */ true, /* blank */ true, /* passphrase */ "", /* avoid_reuse */ false, /* descriptors */ true, /* load_on_startup */ true).flatMap(_ => { - logger.info(s"importing new descriptors ${descriptors(0).descriptors}") - rpcClient.invoke("importdescriptors", descriptors(0).descriptors).collect { - case JArray(results) => results.forall(item => { - val JBool(success) = item \ "success" - success - }) - } - }).recover { e => - logger.error("cannot create eclair-backed on-chain wallet: ", e) - false - } - } - } - override def descriptors(account: Long): Descriptors = { // TODO: we should use 'h' everywhere once bitcoin-kmp supports it. val keyPath = s"$rootPath/$account'".replace('\'', 'h') diff --git a/eclair-core/src/main/scala/fr/acinq/eclair/crypto/keymanager/OnChainKeyManager.scala b/eclair-core/src/main/scala/fr/acinq/eclair/crypto/keymanager/OnChainKeyManager.scala index 969253a803..55ba8db41e 100644 --- a/eclair-core/src/main/scala/fr/acinq/eclair/crypto/keymanager/OnChainKeyManager.scala +++ b/eclair-core/src/main/scala/fr/acinq/eclair/crypto/keymanager/OnChainKeyManager.scala @@ -29,17 +29,6 @@ import scala.util.Try trait OnChainKeyManager { def walletName: String - /** - * @return the creation time of the wallet managed by this key manager - */ - def walletTimestamp(): TimestampSecond - - /** - * Create a bitcoin core watch-only wallet with private keys owned by this key manager instance. - * This should only be called if the corresponding wallet doesn't already exist. - */ - def createWallet(rpcClient: BitcoinJsonRPCClient)(implicit ec: ExecutionContext): Future[Boolean] - /** * @param account account number (0 is used by most wallets) * @return the on-chain pubkey for this account, which can then be imported into a BIP39-compatible wallet such as Electrum diff --git a/eclair-core/src/test/scala/fr/acinq/eclair/blockchain/bitcoind/BitcoinCoreClientSpec.scala b/eclair-core/src/test/scala/fr/acinq/eclair/blockchain/bitcoind/BitcoinCoreClientSpec.scala index 505340a82b..5a38d6ae97 100644 --- a/eclair-core/src/test/scala/fr/acinq/eclair/blockchain/bitcoind/BitcoinCoreClientSpec.scala +++ b/eclair-core/src/test/scala/fr/acinq/eclair/blockchain/bitcoind/BitcoinCoreClientSpec.scala @@ -1418,8 +1418,7 @@ class BitcoinCoreClientWithEclairSignerSpec extends BitcoinCoreClientSpec { val seed = MnemonicCode.toSeed(MnemonicCode.toMnemonics(entropy), "") val master = DeterministicWallet.generate(seed) val (wallet, keyManager) = createWallet(seed) - keyManager.createWallet(wallet.rpcClient).pipeTo(sender.ref) - sender.expectMsg(true) + createEclairBackedWallet(wallet.rpcClient, keyManager) // this account xpub can be used to create a watch-only wallet val accountXPub = DeterministicWallet.encode( @@ -1453,8 +1452,7 @@ class BitcoinCoreClientWithEclairSignerSpec extends BitcoinCoreClientSpec { (1 to 10).foreach { _ => val (wallet, keyManager) = createWallet(randomBytes32()) - keyManager.createWallet(wallet.rpcClient).pipeTo(sender.ref) - sender.expectMsg(true) + createEclairBackedWallet(wallet.rpcClient, keyManager) wallet.getReceiveAddress().pipeTo(sender.ref) val address = sender.expectMsgType[String] diff --git a/eclair-core/src/test/scala/fr/acinq/eclair/blockchain/bitcoind/BitcoindService.scala b/eclair-core/src/test/scala/fr/acinq/eclair/blockchain/bitcoind/BitcoindService.scala index 4aabf798be..8c1d63e283 100644 --- a/eclair-core/src/test/scala/fr/acinq/eclair/blockchain/bitcoind/BitcoindService.scala +++ b/eclair-core/src/test/scala/fr/acinq/eclair/blockchain/bitcoind/BitcoindService.scala @@ -25,7 +25,7 @@ import fr.acinq.bitcoin.scalacompat.{Block, Btc, BtcAmount, MilliBtc, MnemonicCo import fr.acinq.eclair.blockchain.bitcoind.rpc.BitcoinJsonRPCAuthMethod.{SafeCookie, UserPassword} import fr.acinq.eclair.blockchain.bitcoind.rpc.{BasicBitcoinJsonRPCClient, BitcoinCoreClient, BitcoinJsonRPCAuthMethod, BitcoinJsonRPCClient} import fr.acinq.eclair.blockchain.fee.{FeeratePerByte, FeeratePerKB, FeeratePerKw} -import fr.acinq.eclair.crypto.keymanager.LocalOnChainKeyManager +import fr.acinq.eclair.crypto.keymanager.{LocalOnChainKeyManager, OnChainKeyManager} import fr.acinq.eclair.integration.IntegrationSpec import fr.acinq.eclair.{BlockHeight, TestUtils, TimestampSecond, randomKey} import grizzled.slf4j.Logging @@ -171,8 +171,7 @@ trait BitcoindService extends Logging { val sender = TestProbe() waitForBitcoindUp(sender) if (useEclairSigner) { - onChainKeyManager.createWallet(bitcoinrpcclient).pipeTo(sender.ref) - sender.expectMsg(true) + createEclairBackedWallet(bitcoinrpcclient, onChainKeyManager) } else { sender.send(bitcoincli, BitcoinReq("createwallet", defaultWallet)) sender.expectMsgType[JValue] @@ -182,6 +181,17 @@ trait BitcoindService extends Logging { awaitCond(currentBlockHeight(sender) >= BlockHeight(150), max = 3 minutes, interval = 2 second) } + def createEclairBackedWallet(bitcoinClient: BitcoinJsonRPCClient, keyManager: OnChainKeyManager): Unit = { + val sender = TestProbe() + // wallet_name, disable_private_keys, blank, passphrase, avoid_reuse, descriptors, load_on_startup, external_signer + bitcoinClient.invoke("createwallet", keyManager.walletName, true, false, "", false, true, true, false).pipeTo(sender.ref) + sender.expectMsgType[JValue] + val descriptors = keyManager.descriptors(0).descriptors + bitcoinClient.invoke("importdescriptors", descriptors).pipeTo(sender.ref) + sender.expectMsgType[JValue] + } + + /** Generate blocks to a given address, or to our wallet if no address is provided. */ def generateBlocks(blockCount: Int, address: Option[String] = None, timeout: FiniteDuration = 10 seconds)(implicit system: ActorSystem): Unit = { val sender = TestProbe() diff --git a/eclair-core/src/test/scala/fr/acinq/eclair/channel/publish/ReplaceableTxPublisherSpec.scala b/eclair-core/src/test/scala/fr/acinq/eclair/channel/publish/ReplaceableTxPublisherSpec.scala index 6245b5f5db..1d75d4390e 100644 --- a/eclair-core/src/test/scala/fr/acinq/eclair/channel/publish/ReplaceableTxPublisherSpec.scala +++ b/eclair-core/src/test/scala/fr/acinq/eclair/channel/publish/ReplaceableTxPublisherSpec.scala @@ -1665,8 +1665,7 @@ class ReplaceableTxPublisherWithEclairSignerSpec extends ReplaceableTxPublisherS override def getP2wpkhPubkey(renew: Boolean): PublicKey = pubkey } - keyManager.createWallet(walletRpcClient).pipeTo(probe.ref) - probe.expectMsg(true) + createEclairBackedWallet(walletRpcClient, keyManager) (walletRpcClient, walletClient) } From d3ac58863fbb76f4a44a779a52a6893b43566b29 Mon Sep 17 00:00:00 2001 From: sstone Date: Thu, 21 Sep 2023 15:06:45 +0200 Subject: [PATCH 8/8] Document how we setup an optional onchain key manager Depending upon the presence of an `eclair-signer.conf` file in Eclair's data directory, and the names of the wallet set in `eclair-signer.conf` and `eclair.conf`, we can have: - no onchain key manager (which is the default) - an onchain key manager that is used to generate descriptors through our API but that is not active. This is how you create a new watch-only Bitcoin wallet to be used by Eclair - an onchain key manager that is used to sign bitcoin wallet transactions --- eclair-core/src/main/scala/fr/acinq/eclair/Setup.scala | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/eclair-core/src/main/scala/fr/acinq/eclair/Setup.scala b/eclair-core/src/main/scala/fr/acinq/eclair/Setup.scala index d52eb8860d..98eb355737 100644 --- a/eclair-core/src/main/scala/fr/acinq/eclair/Setup.scala +++ b/eclair-core/src/main/scala/fr/acinq/eclair/Setup.scala @@ -121,6 +121,7 @@ class Setup(val datadir: File, // early checks PortChecker.checkAvailable(serverBindingAddress) + // load on onchain key manager if an `eclair-signer.conf` is found in Eclair's data directory val onChainKeyManager_opt = LocalOnChainKeyManager.load(datadir, NodeParams.hashFromChain(chain)) val (bitcoin, bitcoinChainHash) = { @@ -247,6 +248,13 @@ class Setup(val datadir: File, finalPubkey = new AtomicReference[PublicKey](null) pubkeyRefreshDelay = FiniteDuration(config.getDuration("bitcoind.final-pubkey-refresh-delay").getSeconds, TimeUnit.SECONDS) + // there are 3 possibilities regarding onchain key management: + // 1) there is no `eclair-signer.conf` file in Eclair's data directory, Eclair will not manage Bitcoin core keys, and Eclair's API will not return bitcoin core descriptors. This is the default mode. + // 2) there is an `eclair-signer.conf` file in Eclair's data directory, but the name of the wallet set in `eclair-signer.conf` does not match the `eclair.bitcoind.wallet` setting in `eclair.conf`. + // Eclair will use the wallet set in `eclair.conf` and will not manage Bitcoin core keys (here we don't set an optional onchain key manager in our bitcoin client) BUT its API will return bitcoin core descriptors. + // This is how you would create a new bitcoin wallet whose private keys are managed by Eclair. + // 3) there is an `eclair-signer.conf` file in Eclair's data directory, and the name of the wallet set in `eclair-signer.conf` matches the `eclair.bitcoind.wallet` setting in `eclair.conf`. + // Eclair will assume that this is a watch-only bitcoin wallet that has been created from descriptors generated by Eclair, and will manage its private keys, and here we pass the onchain key manager to our bitcoin client. bitcoinClient = new BitcoinCoreClient(bitcoin, if (bitcoin.wallet == onChainKeyManager_opt.map(_.walletName)) onChainKeyManager_opt else None) with OnchainPubkeyCache { val refresher: typed.ActorRef[OnchainPubkeyRefresher.Command] = system.spawn(Behaviors.supervise(OnchainPubkeyRefresher(this, finalPubkey, pubkeyRefreshDelay)).onFailure(typed.SupervisorStrategy.restart), name = "onchain-address-manager")