On Mon, Nov 25, 2019 at 8:38 PM Dmitry Timoshkov dmitry@baikal.ru wrote:
Jeff Smith whydoubt@gmail.com wrote:
- char data[256];
- char data[2048]; /* 2 x 2 pixels, max 64 bits-per-pixel, max 64 planes */
Using data from your original patch I get 2544 bytes.
If you plug the numbers into a standard calculator, that is what you would get. However, this is integer math, so...
((width * bpp + 31) / 32) * 4 * height * planes ((2 * 64 + 31) / 32) * 4 * 2 * 64 ((128 + 31) / 32) * 4 * 2 * 64 <---- 128 bits to hold data for 1 row of pixels (159 / 32) * 4 * 2 * 64 4 * 4 * 2 * 64 16 * 2 * 64 <---- 16 byte stride (1 row of pixels, padded to 4-byte boundary) 32 * 64 <---- 32 bytes for a single plane 2048 <---- 2048 bytes for all planes
My comment in the latest patch does gloss over the difference between between line width and line stride, which is effectively identical in this case.
Jeff