Until now, all the examples of Web3.js library's sendTransaction() method we saw were using the from address that's present in the Ethereum node; therefore, the Ethereum node was able to sign the transactions before broadcasting. But if you have the private key of a wallet stored somewhere else, then geth cannot find it. Therefore, in this case, you will need to use the web3.eth.sendRawTransaction() method to broadcast transactions.

web3.eth.sendRawTransaction() is used to broadcast raw transactions, that is, you will have to write code to create and sign raw transactions. The Ethereum node will directly broadcast it without doing anything else to the transaction. But writing code to broadcast transactions using web3.eth.sendRawTransaction() is difficult because it requires generating the data part, creating raw transactions, and also signing the transactions.

The Hooked-Web3-Provider library provides us with a custom provider, which communicates with geth using HTTP; but the uniqueness of this provider is that it lets us sign the sendTransaction() calls of contract instances using our keys. Therefore, we don't need to create data part of the transactions anymore. The custom provider actually overrides the implementation of the web3.eth.sendTransaction() method. So basically, it lets us sign both the sendTransaction() calls of contract instances and also the web3.eth.sendTransaction() calls. The sendTransaction() method of contract instances internally generate data of the transaction and calls web3.eth.sendTransaction() to broadcast the transaction.

EthereumJS is a collection of those libraries related to Ethereum. ethereumjs-tx is one of those that provide various APIs related to transactions. For example, it lets us create raw transactions, sign the raw transactions, check whether transactions are signed using proper keys or not, and so on.

Both of these libraries are available for Node.js and client-side JavaScript. Download the Hooked-Web3-Provider from https://www.npmjs.com/package/hooked-web3-provider, and download ethereumjs-tx from https://www.npmjs.com/package/ethereumjs-tx.

At the time of writing this book, the latest version of Hooked-Web3-Provider is 1.0.0 and the latest version of ethereumjs-tx is 1.1.4.

Let's see how to use these libraries together to send a transaction from an account that's not managed by geth.

var provider = new HookedWeb3Provider({ 
 host: "http://localhost:8545", 
 transaction_signer: { 
  hasAddress: function(address, callback){ 
   callback(null, true); 
  }, 
  signTransaction: function(tx_params, callback){ 
   var rawTx = { 
          gasPrice: web3.toHex(tx_params.gasPrice), 
          gasLimit: web3.toHex(tx_params.gas), 
          value: web3.toHex(tx_params.value) 
          from: tx_params.from, 
          to: tx_params.to, 
          nonce: web3.toHex(tx_params.nonce) 
      }; 

   var privateKey = EthJS.Util.toBuffer('0x1a56e47492bf3df9c9563fa7f66e4e032c661de9d68c3f36f358e6bc9a9f69f2', 'hex'); 
   var tx = new EthJS.Tx(rawTx); 
   tx.sign(privateKey); 

   callback(null, tx.serialize().toString('hex')); 
  } 
 } 
}); 

var web3 = new Web3(provider); 

web3.eth.sendTransaction({ 
 from: "0xba6406ddf8817620393ab1310ab4d0c2deda714d", 
 to: "0x2bdbec0ccd70307a00c66de02789e394c2c7d549", 
 value: web3.toWei("0.1", "ether"), 
 gasPrice: "20000000000", 
 gas: "21000" 
}, function(error, result){ 
 console.log(error, result) 
})

Here is how the code works:

1. At first, we created a HookedWeb3Provider instance. This is provided by the Hooked-Web3-Provider library. This constructor takes an object that has two properties, which must be provided. host is the HTTP URL of the node and transaction_signer is an object that the custom provider communicates with to get the transaction signed.
2. The transaction_signer object has two properties: hasAddress and signTransaction. hasAddress is invoked to check whether the transaction can be signed, that is, to check whether the transaction signer has the private key of the from address account. This method receives the address and a callback. The callback should be called with the first argument as an error message and the second argument as false if the private key of the address is not found. And if the private key is found, the first argument should be null, and the second argument should be true.

3. If the private key for the address is found, then the custom provider invokes the signTransaction method to get the transaction signed. This method has two parameters, that is, the transactions parameters and a callback. Inside the method, at first, we convert the transaction parameters to raw transaction parameters, that is, the raw transaction parameters values are encoded as hexadecimal strings. Then we create a buffer to hold the private key. The buffer is created using the EthJS.Util.toBuffer() method, which is part of the ethereumjs-util library. The ethereumjs-util library is imported by the ethereumjs-tx library. We then create a raw transaction and sign it, after which we serialize and convert it to hexadecimal strings. Finally, we need to provide the hexadecimal string of the signed raw transaction to the custom provider using the callback. In case there is an error inside the method, then the first argument of the callback should be an error message.
4. Now the custom provider takes the raw transactions and broadcasts it using web3.eth.sendRawTransaction().
5. Finally, we call the web3.eth.sendTransaction function to send some ether to another account. Here, we need to provide all the transaction parameters except nonce because the custom provider can calculate nonce. Earlier, many of these were optional because we were leaving it to the Ethereum node to calculate them, but now as we are signing it ourselves, we need to provide all of them. The gas is always 21,000 when the transaction doesn't have any data associated with it.