Tight encoding with zlib compression level 0 (uncompressed)

[PT2020-cd]

When using Tight encoding with zlib compression level 0 (uncompressed), the libvncserver is bypassing the zlib compression and sending the uncompressed buffer. This is not working on clients UltraVNCViewer and TightVNC. Solution: compress the buffer with zlib level 0 before sending the data. The difference is that the zlib header will be attached before the buffer even if there is no compression.

Following lines have to be removed in libvncserver/tight.c (I set them in comment):

SendMonoRect:

    dataLen = (w + 7) / 8;
    dataLen *= h;

/*    if (tightConf[compressLevel].monoZlibLevel == 0 &&
        cl->tightEncoding != rfbEncodingTightPng)
        cl->updateBuf[cl->ublen++] =
            (char)((rfbTightNoZlib | rfbTightExplicitFilter) << 4);
    else
*/        cl->updateBuf[cl->ublen++] = (streamId | rfbTightExplicitFilter) << 4;
    cl->updateBuf[cl->ublen++] = rfbTightFilterPalette;
    cl->updateBuf[cl->ublen++] = 1;

SendIndexedRect:

    /* Prepare tight encoding header. */
/*    if (tightConf[compressLevel].idxZlibLevel == 0 &&  cl->tightEncoding != rfbEncodingTightPng) {
    	cl->updateBuf[cl->ublen++] =
            (char)((rfbTightNoZlib | rfbTightExplicitFilter) << 4);
    } else
*/        cl->updateBuf[cl->ublen++] = (streamId | rfbTightExplicitFilter) << 4;
    cl->updateBuf[cl->ublen++] = rfbTightFilterPalette;
    cl->updateBuf[cl->ublen++] = (char)(paletteNumColors - 1);

SendFullColorRect:

    if (cl->ublen + TIGHT_MIN_TO_COMPRESS + 1 > UPDATE_BUF_SIZE) {
        if (!rfbSendUpdateBuf(cl))
            return FALSE;
    }

/*    if (tightConf[compressLevel].rawZlibLevel == 0 &&
        cl->tightEncoding != rfbEncodingTightPng)
        cl->updateBuf[cl->ublen++] = (char)(rfbTightNoZlib << 4);
    else
*/        cl->updateBuf[cl->ublen++] = 0x00;  /* stream id = 0, no flushing, no filter */
    rfbStatRecordEncodingSentAdd(cl, cl->tightEncoding, 1);

CompressData:

/*    if (zlibLevel == 0) {
    	printf("VNC: UnComp %i\n",dataLen);
        return rfbSendCompressedDataTight(cl, tightBeforeBuf, dataLen);
    }
*/
    pz = &cl->zsStruct[streamId];

[bk138]

By filing a pull request, we can keep credit and your authorship, what do you think?

[PT2020-cd]

It depends on what this means. How do I file a pull request?
But to keep it easy you can find someone who can confirm the problem and the fix. You also can use the fix without crediting me so it will help the community.

[StefanBruens]

@PT2020-cd - you can "Fork" this repository (button on the top right), and then edit the file in your fork.

When you are done editing, "commit" the changes to your repository, adding an appropriate commit message what your change is about (Don't forget to reference this issue, having "#370" in the commit message is sufficient).

At last, you create a merge request.

[marcofortina]

I opened a PR for this issue:

#712

It removes the rfbTightNoZlib shortcuts for mono, indexed and full-color Tight rectangles when the selected zlib compression level is 0, and also removes the zlibLevel == 0 bypass in CompressData() so the payload is still emitted as a valid zlib stream.

Local validation completed:

100% tests passed, 0 tests failed out of 5