VisualSFM Tutorial

Visual SFM > Meshlab Workflow

I will generally be comparing the shared source photogrammetry software VisualSFM to Agisoft Photoscan. While there are many other free and open sourced photogrammetry software as well as proprietary options that exist, I have the most experience with these two programs. These two programs are also some of the most widely used programs for photogrammetry. Where the open/shared source software that I have been using is currently, this workflow seems better suited for educational purposes such as providing poeple with a free and relatively simple way to produce photogrammetric models at home or elsewhere. While the VisualSFM to Meshlab workflow produces visually accurate models of the original object or space, it does not allow the level of fine tuning and metering of point reconstruction accuracy as Agisoft. However, with the VisualSFM to Meshlab workflow, you can create a textured mesh from images in a short amount of time.

VisualSFM Overview

VisualSFM is a GUI application for 3D reconstruction using structure from motion (SFM). You can download this package http://opensourcephotogrammetry.blogspot.com/2010/12/new-package-available-for-windows.html that includes the libraries for feature detection and matching, sparse reconstruction and dense reconstruction typically used in VisualSFM. The .ply files created here can then be opened in Meshlab and converted into textured 3D meshes. VisualSFM is a free, shared-source (http://ccwu.me/vsfm/README) application created by Changchang Wu.

Download VisualSFM

Limitations of VisualSFM

VisualSFM will only work on textured objects. Uniform surfaces such as a white wall will not work well. This tool will not work well if too few features are detected.

VisualSFM only accepts jpeg, ppm, and pgm image file types The default maximum working dimension for images is 3200. If you have enough memory for size 4000, change the parameter by going to Tools -> Enable GPU -> Set Maximum DIM from the main tool bar at the top of the interface.

Tips:

When you download the package from the link above, make sure the CMVS/PMVS binaries for sparse and dense reconstruction are placed in the same location as the VisualSFM.exe within the VisualSFM folder created upon downloading.

Minimize zooming, mouse dragging. etc. while processing to reduce the chance of memory leak (rare but possible) that may cause information to be lost. It is best to minimize the window during processing.

1) Open VisualSFM. Then open the image set you want to use by selecting the open-multi images icon and locating the images.

2) Once all images have been uploaded, go to Tools -> Enable GPU -> Set Maximum Dim. The default is 3200 pixels. You can raise the maximum dimensions if your using higher resolution images in order to get for information from the photos. Otherwise, VisualSFM will automatically down res your images. This might be neccesary depending on the amount of memory your computer can use.

3) Select the Compute Missing Matches icon from the tool bar to begin feature detection and matching.

4) Once the matching has finished, select the Compute 3D Reconstruction icon from the tool bar to begin the sparse reconstruction.

5) Select the CMVS icon to begin the dense reconstruction. You will be prompted to save out the folder that will be created during the dense reconstruction inside your VisualSFM folder as a nvm.cmvs file.

A finished dense reconstruction. You can toggle between the sparse and dense reconstruction view at any point using the tab key. Once the dense reconstruction has finished. It will be automatically saved in the folder you created. VisualSFM can be closed and your ready for Meshlab!

Meshlab

Meshlab is a free and open source software designed for editing unstructured meshes typically produced by 3D scans. We will be using Meshlab to transform our point clouds into textured meshes.

Meshlab tip: There is no undo option in Meshlab. Instead, it is a good idea to export the current version of project after each step. You can go to File -> Reload to recover a previous unsaved state if you want to undo a deletion for example.

Download Meshlab

is a helpful Meshlab tutorial for info in scaling and other actions.

Here

1) Open Meshlab. Go to File ->Open Project and navigate to the .nvm.cmvs folder generated in VisualSFM. That should bring you to a folder labeled 00. Select the bundle.rd.out file. Next, you will be prompted to select a text file named list.txt. Select this.

Once the sparse point cloud has been imported, go to Render -> Show Cameras. The cameras may need to be scaled down. Click on the layer icon to show the layers panel on the right side of the screen. Open the drop down menu where it says Show cameras and change the scale from 5 to .0001 or until you can see point cloud.

2) Go to File-> Import Meshes. Navigate to the models folder in the same nvm.cmvs folder. Choose the .ply file in this folder. There may be multiple .ply files for larger point clouds. Import all .plys at the same time. These are your dense reconstruction.

3) Now you can go in and clean the dense point clouds. The sparse reconstruction is only needed for camera placement, so from now on we will only be working with the dense point cloud.

Select the Delete Selected Vertices icon to clear your selection.

4) Go to Filters -> Normals, Curvatures, and Orientation -> Compute normals for point set.

Meshing the point cloud

5) If your dense point cloud is divided into multiple .ply files, turn on all .ply layers, turn off or delete the sparse point cloud layer, right click any visible layer and select Flatten Visible layers. This will merge you dense point clouds into one. This will also remove the color information from the point clouds.

6) Select the Show Current Raster Mode icon to check if the cameras are aligned correctly to the dense reconstruction. If the alignment is way off, the error probably occured in VisualSFM, and you may want to regenerate the point cloud.

7) Now you can generate your mesh from the point cloud. Make sure that your dense point cloud layer (misleadingly named mesh or merged mesh) is highlighted. Go to Filters -> Point Set -> Surface reconstruction: Poisson.

I'm still working out the meaning of the different setting but generally...

The Octree Depth controls the resolution of the triangulated mesh. Increasing this amount will increase the number of divisions of the mesh, creating more detail. Depending on the level of detail of your model, you probably want to increase the Octree Depth from the low resolution default amount of 6. However, increasing the Octree Depth uses more memory, slowing down the program. Increasing the Solver Divide helps to save some memory as the Octree Depth is increased. Increasing the Samples per Node smooths out your mesh if you want. Surface offsetting can help to remove some unwanted artifacts of the meshing process.

Hit apply when done.

8) Clean up the resulting mesh if needed using the Face Selection icon and the Delete Vertices icon to select and remove unwanted parts of the mesh.

9) The model has to be manifold in order to build the texture. Go to Filters -> Selection -> Select non-manifold edges. Clear the selection by clicking the Delete Vertices icon in the toolbar.

Texturing the Mesh

10) Go to Filters -> Texture -> Paramaterization from registered rasters.

11) Then go to Filters -> Texture -> Project active rasters color to current mesh, filling the texture. from this step you should get the color information projected onto the mesh, not the final texture. A color file will automatically be saved and exported.

12) Next, to to Filters -> Texture -> Paramaterization + Texturing from registered rasters. Once you've finished this step you should see the texture projected into the mesh. The texture file will automatically be saved and exported.

Change the pixel size of texture image to whatever size you want to work with. I have just left the rest of the default settings as is.