Silicon Pr0n

This page is read only. You can view the source, but not change it. Ask your administrator if you think this is wrong.
Confocal Imaging Computer Vision (CFCV) tookit

This page is about the pr0nsweeper Python GUI that allows a human to quickly review and fix CV results.

Proof of concept stage / not intended for general use.


====== Keyboard and mouse quick summary ======

Left click: metal operation
  * Void => metal
  * Unknown => metal
  * Metal => unknown

Right click: void operation
  * Unknown => void
  * Metal => void
  * Void => unknown

Shift click: do above operation on all adjacent tiles (self inclusive) of the same tile type

Enter/return: submit

Escape: submit but flag for administrative review.  Use when CV failed beyond reasonable human correction (ex: grid detected very poorly)

M: convert all unknowns to blue metal

W: convert all unknowns to white void


====== Introduction ======

An integrated circuit is composed of metal connecting various areas together to form nets.  We want to recover netlists from various chips but so far all efforts have revolved around manually drawing polygons from microscope images.  A few people have tried to do some computer vision (CV) approaches but ultimately fixing the errors takes longer than drawing the original polygons.

Can we do better?  Sort of...we can cheat ("cheating is a valid engineering strategy").  We cheat by using a confocal microscope (or other high contrast imager) on a chip laid out in a grid.  This simplifies the problem dramatically:
  * The system is now regenerative: errors can be corrected
  * Grid => simple algorithms determine if metal or not in that square

However, this computer vision process is still far from perfect and generates a lot of uncertainties.  This GUI presents the CV results to a human that can quickly correct minor CV errors (humans are slow but much better at pattern matching than computers).  The system tries to flag areas for review rather than making a concrete decision if its unsure.  It may from time to time entirely misinterpret an area.  Your job is to look for and correct these errors.


====== GUI overview ======

{{:pr0ntools:cfcv:pr0nsweeper:overview.png?300|}}

At startup, the left area has the raw CV result and the right has the original image with the CV result overlaid on top.  You'll notice there are a few different tile colors on the left:
  * Blue metal: "wires"
  * White void: no material present
  * Orange unknown: could not decide if metal or void

In most images there will be a large amount of blue and white with a number of orange tiles.  The typical workflow is to correct a small number of errors (if any) and then convert all remaining warnings to metal.  These steps are described in more detail in the following sections.

The reason why most warnings are metal is because the thresholding algorithm currently used has a hard time dealing with vias.  Future versions will hopefully handle them better.


====== Clicking ======

Left clicking is metal oriented: it turns tiles into metal.  if the tile is already metal, it turns it into unknown.  Similarly, right clicking is void oriented.

You may click on either the grid on the left or the image on the right.


{{:pr0ntools:cfcv:pr0nsweeper:group-pre.png?100}}

{{:pr0ntools:cfcv:pr0nsweeper:group-post.png?100}}

Above: before and after shift right clicking to remove entire upper left unknown group

Sometimes its useful to change entire clusters at once.  This can be accomplished by shift clicking.


====== Global operations ======

{{:pr0ntools:cfcv:pr0nsweeper:global_m_pre.png?150}}

{{:pr0ntools:cfcv:pr0nsweeper:global_m_post.png?150}}

Above: using "M" to turn all unknowns into metal

Often the entire remaining unknowns will below to one type or another.  Hit "M" to change all unknowns to metal or "W" to change all unknowns to void.


====== Grid errors ======

{{:pr0ntools:cfcv:pr0nsweeper:grid_error.png?450}}

Above: Y grid error visible in the upper left and lower right corner

If the grid is detected incorrectly it will disrupt metal detection.  In the above example, the Y grid offset is very poor (although the pitch is correct).  Correct errors like above by biasing the grid to a particular direction, using the same bias for the entire image.  For example, make the lower unknowns metal and the upper unknowns void.  Suggest using this convention (lower => metal, upper => void) if its unclear which direction the correction should be applied.

For major errors (ex: grid detection failed) press Escape to skip this tile.  It will be reviewed by an administrator.


====== Splash page ======

{{:pr0ntools:cfcv:pr0nsweeper:splash.png?300}}

If you see this the server has no jobs available

NOTE: the splash page can be edited with standard mouse and keyboard shortcuts.  Attempting to submit discards changes


====== Roadmap ======

The GUI frontend will become some sort of web app so that it can be used from a standard browser without installing any software.

There's still a lot that can be done with the computer vision preprocessing.  The current suite is somewhat optimized for the references images: to be seen how well it will apply to other data sets (ex: SEM images).

====== Running ======

Ubuntu 12.04x64

sudo apt-get install -y python-qt4 python-imaging python-pyicu

git clone https://github.com/JohnDMcMaster/pr0ntools.git

cd cf/sweeper

python sweeper.py --host sketchyurl.com

====== Backend ======

This is for informational purposes only.  Its not required to use the pr0nsweeper GUI.

Data is collected using [[mcmaster:tech_inst:k2_ind|my confocal microscope]].  It is not stitched or preprocessed in any way.

The core CV engine is in cfcv.py.  This basically performs the following steps:
  * Straighten image
  * Determine design grid
  * Calculate metal and void thresholds
  * Apply thresholds to image
  * Munge output to try to heuristically correct errors

cfcv_batch.py is a batch processor around the engine that allows, for example, running jobs in parallel.

Once cfcv_batch.py completes the output directory is fed into the pr0nsweeper job distribution server, server.py.  When a client requests an image it grabs one from the output directory and feeds it to the client.  The client suggests changes to the image and then sends it back to the server where the results are recorded.


====== References ======

  * [[https://github.com/JohnDMcMaster/pr0ntools/tree/master/capture/cf|Source code]]