It is an asset / windfall and, as such, payable under the terms of your IVA. The fact that the holiday was purchased prior to the IVA is immaterial -- say, for instance, you had a valuable oil painting on the wall that you have had since you were a child .... that would be considered an asset to be sold to pay debts, even though you owned it long before the IVA and any money issues. I am afraid that the same goes for the holiday money.
You can dispute it, but I doubt that CF would back down and I can't see a complaint being upheld, but you might get a sympathetic ear if you are lucky.
My opinions are merely that .. opinions based on experience. Always seek professional advice.
IVA Completed 23rd July 2013 .... C.C. 10th January 2014