PDF Editor: Bug Fixing and Flattened PDF Export

And as a small bonus: Triangulation functions

Images in PDF Annotations

Excalidraw supports including images (and potentially other files?) in scenes.
When testing this feature in my PDF Editor/Annotator, I ran into a nasty problem:
For some reason, loading an embedded scene from an SVG does not correctly handle included files.

After a lot of testing and back and forth, I was finally able to mitigate this issues by transforming the files returned from the Excalidraw export function (loadFromBlob) into an array and adding them using the addFiles function of the ExcalidrawAPI.

It was very frustrated by this issue and felt like it took me too long to identify the root-cause. The internal open-file mechanism of Excalidraw worked fine, so I first suspected some of my code caused the issue (and that the loadFromBlob does, in fact, add all files to the scene).

After all, I am of course happy to fix was so issue to implement and that everything appears to work smoothly now.

New Saving Mechanism

When I started integrating CKEditor (a web-based rich-text editor) into my note-taking app, I struggled to correctly pass data from child-components to parent components in React. Everywhere I looked only, using Refs (and often times also class components) was commonly recommended.

I implemented an useImperativeHandle-hook-based solution for the editor, was a bit frustrated with the experience but was a bit desperate for a working solution, so I stuck with.

With time, I implemented multiple other Components from which I wanted to (on-command by the parent component) receive data from.

With a particular annoying case (my already overly complicated previous PDF Viewer), I opted for simply attaching functions to the global window object.
This worked fine, and I had a few mechanisms in place (like using a random string ID as key to retrieve the function), but I was already well aware that this would be considered a dirty hack.

Now, finally, I thought about what I learned so far and how I could implement a different system for getting data from the child components and saving the note content.
I opted for an approach of passing register functions down to the child components.
Those will be called by the child component (e.g., in a useEffect) and then execute the corresponding register function in the parent.
This register function gets a function for requesting the current data or content from the child component passed as an argument.
The parent the saves this function (for example, as state) and can then call this function whenever it requires data from the child.

This is still a simple approach. I wondered why I did not opt for this solution when first encountering the problem. I think I simply did not think about passing functions as props that much (and especially not higher-order functions).

I still need to implement this mechanism for the different child (editor) components, so far, I only changed the new PDF Editor.
But as a consequence: Saving PDF data (the PDF itself and annotations) now works nicely!

Furthermore, I could directly make use of this feature (which also enables the parent to request the PDF itself and annotations separately from the PDF Editor child component) to implement exporting annotated PDFs.

PDF Annotation Export

I had an export function for flattened PDFs for my old PDF Viewer, which had quite a few issues (and was a bit hacky).

To export a PDF, I need to add all included annotations to the corresponding page.

I can use the exported Excalidraw SVG, which I already save for the annotations anyway (in fact, the whole Excalidraw scene is embedded in it)
A fork of the pdf-lib library then allows drawing SVGs on the page.
But as I also (want to) support drawing outside the page bounds of the PDF, I need to do some heavy lifting (or better: juggling) of the page dimension, the annotation dimension, the scaling factor, the margins, translating the original page content, etc.

With my old export function as a starting point, I had the basic export code written out rather quickly. Getting all the dimensions and offsets right, was a bit harder (especially if you have to constantly remember that the origin is considered to be in the top-left corner for some functions).

After some more time, it was almost working perfectly. But for some PDFs and some pages of these PDFs, I noticed some subtle alignment issues (i.e., annotations were drawn too high or low on the page).
Through some experimenting, I identified the BleedBox of the pages to be responsible for these differences, considering its y component when drawing the SVG yielded (almost?) perfect results. Nice!

I still have to do a lot of testing, but so far, the PDFs I tried, were all exported nicely.

But there were some other issues I was already aware of, as I already noticed them in my old export function:

  1. Included images in the annotations (see above) are not exported at all
  2. Rotated text (and, as I later found out, also images) are not rotated in the exported PDF
  3. Fonts are not correctly embedded and displayed

To address these issues, I had to deep-dive into the exported SVG elements from Excalidraw, inspect the code of the pdf-lib fork to see how which attributes are handled, and even had to reverse-engineer the offset of rotated images by wildly guessing triangulation functions that could result in the right value.

For that, I basically wrote an SVG-preprocessor function, which transforms the exported Excalidraw annotation SVG to an SVG which can be nicely embedded into the PDF file using the pdf-lib fork.

The rotated texts and images were especially tricky, as the rotation in the SVG-transformation property can define an origin in addition to the rotation angle.
The pdf-lib fork does not support this, so I had to experiment and get the help of Wolfram Alpha to correctly transform the image position back using the x and y properties.

Below you can see the result: The original annotations on the PDF (left) match the exported PDF (right), including the rotation and size of images!

Screenshot: On the left, a browser with the PDF Editor open. A PDF with rotated images and text is displayed. One of the texts says "Thanks WolframAlpha!".
On the right: the exported PDF in a PDF viewer, showing a (nearly) identical PDF.
Finally Working: Rotated text and image elements from excalidraw are correctly rotated and displayed in exported PDF file.

I also included the part of the source code of the SVG-preprocessing function, which calculates the correct image x and y attributes based on the identified rotation angle.

Keep in mind, that this is the original code, when I first got the rotation-fixes working correctly. I wanted to make sure to include all reverse-engineering and experimentation bits and pieces still present in the code.

Handling Rotated Exaclidraw Elements for SVG-based PDF Export
  for(const el of svgObj.querySelectorAll('text, image')){
    const textParentEl = el.parentElement;
    if(textParentEl && textParentEl.tagName === 'g' && 'transform' in textParentEl){
      const  trans : SVGAnimatedTransformList =  textParentEl.transform as any;
      let angle: number|undefined = undefined;
      for(let i = 0; (i < trans.baseVal.length && angle === undefined); i++){
        if(trans.baseVal.getItem(i).type ===  trans.baseVal.getItem(i).SVG_TRANSFORM_ROTATE){
          angle = trans.baseVal.getItem(i).angle;
        }
      }
      if(angle !== undefined){
        el.setAttribute('rotate',(-angle).toString())
        if(el.tagName === 'image'){
          console.log(angle)
          const v = 176.889;
          const heightStr = el.getAttribute('height');
          const height = parseFloat(heightStr || "100");
          const alpha = 45.10598;
          el.setAttribute('x',(Math.sin(degreesToRadians(angle)) * -height).toString())
          el.setAttribute('y',((1-Math.cos(degreesToRadians(angle))) * height).toString())

          
          // el.setAttribute('x',(-250).toString()) // -cos(alpha) * d
          // el.setAttribute('y',(110).toString()) //maybe (1-cos(alpha)) *d


          // w=345
        // angle: 352
          // el.setAttribute('x',(48).toString())  // -sin(alpha) * d
          // el.setAttribute('y',(5).toString()) 
          // 0.3107
          // el.setAttribute('y',(v).toString())
        }
      }
    }

  }
I intentionally included all comments to showcase my reverse-engineering efforts 😉

Posted

in