I wanted the item you carry around to always follow the player camera, but not perfectly. I wanted it to fall behind and catch up, have its own momentum. Parenting it to the camera would be easy, but it would be stiff. Keeping it independent and constantly Lerping to catch up works, and is what I did in the original, but this introduces a lot of unpleasant jitter. This is a conundrum I've come to know as "gun-lag" (in a typical first-person game, the item you're carrying would be a gun).

The solution I came up with to improve on the original is averaging the Quaternions. A quaternion, for simplicity's sake, is a rotation state in world space. The current Quaternion is where my object is now (probably lagging behind the camera). And the goal Quaternion is wherever the camera is now (a perfect match between camera and object). Slerping (spherically interpolating) between the current rotation and a goal rotation works, but is jittery, due to imperfect slicing of time and sampling of mouse input. So what if I capture several past orientations of the camera, average them out, then slerp between current and the average instead? 

I keep a running array of 6 stored Quaternions (storedQs[]), I average them, apply the final slerp, and then update the array with a new camera rotation every frame. (Because I couldn't find a legit way to average quaternions, I figured I could just Slerp them one after another. It's a lot of heavy math, and maybe I can optimize it better, but it seems to work for now). Here is the code, which is called every LateUpdate:

    void AvQuatSolution()
    {
        float tdt = Time.deltaTime * snappiness;

        // find average
        Quaternion avQuat = storedQs[0];
        for (int i = 0; i < storedQs.Length - 1; i++) {
            avQuat = Quaternion.Slerp(avQuat, storedQs[i + 1], frontWeight);
        }

        // apply the average to our inventory holder
        transform.rotation = Quaternion.Slerp(transform.rotation, avQuat, tdt);
        
        // shift array for next frame
        for (int i = storedQs.Length - 1; i > 0; i--) {
            storedQs[i] = storedQs[i - 1];
        }
        storedQs[0] = cam.rotation;
    }