You need to firstly distinguish between expenses and disbursements.
The following article may be of help here:
If the contractor is re-charging an expense, then the whole of the expense is booked without any input tax claimed as the consultant is not VAT registered. For e.g. if the consultant is claiming £12 (£10 + £2 vat) worth of fuel used in their car for travelling on company business, then the £12 would all be booked as motor expenses (i.e. you don't/can't claim the £2 as input tax).
If the contractor is re-charging a disbursement that has VAT on it, then the amount ex the VAT is booked and you claim the input tax. So for e.g. if the contractor goes and buys an electric fan for use in the company's office which is for general use (and not just for the consultant's own use), the fan costs £60 (£50 + £10 vat), then £50 is booked as a fixed asset addition and £10 input tax is claimed