This particular error occurs when there's lack of shares in your Demat account of the company offering Buyback. The system verifies that you had the relevant shares in your Demat account on or before the record date for the Buyback.
In simple terms, if you don't possess the required shares by the record date, your Buyback application will be flagged, showing the 'Not enough quantity' error.