What This is All About
With the increasing number of low-cost 3D scanners, 3D capture on smart phones and a wide range of industrial scanners, the opportunities to create value-rich 3D applications are multiple.
Nowadays, many generic software solutions exist for the manufacturing industry: CAD drawing, reverse engineering, quality control of parts, etc. However, there are many gaps left untouched. Many small niches and potential applications for which there is no turn-key solution.
This guide’s focus is primarily on how to develop a 3D application to solve a specific problem. There are so many odd or unknown applications for the industry. Here are some examples that I have been aware of:
- Match a 3D shape within a huge database of similar shapes (example: find the closest shaped steel plate to match a surface to repair or find the best shoe for your foot within an inventory).
- Find the best matching garment for a person’s body shape.
- Repairs. When repairing large or expensive structures, such as ships, pipelines, bridges or airplanes, knowledge is gold. 3D scanners provide a lot of information, but the real gold is when the analysis process is fully automated to make the valuable information readily available on-site.
- Measuring changes of a building or structure overtime (hydroelectric dams, bridges, etc.)
- Volume and surface area measurements done by a LiDAR on a drone.
- Test assembled parts coming from a production chain and send a warning when a problem is predicted to happen in the future, before it happens.
Real life example: During a refinery shutdown, this pressure vessel door has to be put back in place and welded as fast as possible, but the door and the pressure vessel bend right after the cutout (internal stress at work).
Solution: the new pressure vessel shape is recorded and the door can be bent and tested until it matches the shape of the pressure vessel again. The testing fast and simple (automated script in Polyworks) and away from the pressure vessel to let other workers do their job.
Define Your Most Important Criteria
What are your most important criteria? Are you in a competitive niche and worried about your competitors working on the same solution? Is this a one-use scenario or is it possible to use it at many locations? The following list will help you to define your important criteria to make a better development decision.
Being the First to Market
If your application serves a commercial purpose, is it important for you to be the first to market? If you answered yes, then using a high-level programming language is mandatory, and anything that can speed up the development is extremely important. Fast development also means that it will cost you less money to test your ideas. It will also minimize the risk if you are developing an application for a new niche or a new market: you will get feedback from the market faster.
Visualization
Visualization is one of the most underrated criteria, because we mostly see the need when things don’t work as expected. Not all 3D applications require an elegant 3D viewer with hundreds of viewing options. Sometimes, we only need to extract a few numbers from the 3D file and the 3D viewer appears like a waste of time and money.
But visualization is critical during the development phase. If the development environment has poor 3D visualization, critical bugs could be missed. A good trick is to install a free 3D software, such as Blender or FreeCAD, just to make good use of their 3D viewer. Then, you only need to export the results in a generic 3D format to be able to see them.
Hopefully, some solutions offer a great 3D viewer, which is not only a time saver during the development phase, but is also quite useful after the application is delivered, to detect and understand any bug or conceptual mistake.
The Cost of the First Install
This is the total cost for developing the application until you have one working installation. It includes the cost of the development along with all the required licenses for the program.
The Cost per Install
In some cases, the application can depend on a third-party license, and those licenses may have a cost per install or royalty to pay. Some development solutions are royalty-free and open source, but this can be a trap. We often have an idealized view of what we want to achieve: we think that we will sell our application many times and we are reluctant about the idea of buying high-end licenses for each workstation.
Truth is, it is easy, and a bit naive, to aim for the lowest cost per use, multiply by X (enter big number here) because we plan to make a killing, when the priority should be to secure the market, get feedback and market presence as fast as possible.
Personally, I see the cost per install as a priority, but only once the application is already developed and successful.
Safety and Stability
There are situations where the reliability and trustworthiness of the results are more important than anything else. If that is the case, a certified, licensed software with its own scripting editor is by far the best option, if it fits your application needs. Some metrology software can do that, and they have great visualization, which means that it is possible to audit the results each time the application is used.
The Different Working Environments
There are many paths that you can choose when developing a 3D application. However, manipulating 3D scanner data in an industrial environment requires specific characteristics, and you may be surprised by some of the solutions provided, since you may be unaware of them.
Polyworks: the Fastest Software to be the First to Market
Polyworks is a metrology and reverse engineering software made by Innovmetrics. It has a very deep set of functions to automate just about any 3D measurement and alignment of point clouds or meshes. What makes it unique compared to other 3D measurements software is its macro script editor. In Polyworks, it is possible to fully automate a series of measurements. There are added functionalities in the macro script editor that let you accomplish things that cannot be replicated with the user interface. It is a very powerful and underrated solution.
Resources: Polyworks Training and Service Resources
Pros:
Cons:
MATLAB: Great for Mathematically Challenging Projects
MATLAB is a good solution to prototype applications. The environment offers much more mathematical capabilities than Polyworks and it is more adapted to scientific usage.
Pros:
Cons:
Python: a High-level Scripting Language
If you are looking for a free programming language, Python is your best option to build 3D applications. Even if Python is a scripting language, many open sources modules compatible with Python are compiled and optimized. Many are actually made in C++ with Python wrappers. These modules handle the most challenging tasks like the GUI and the mathematics, which means that it is possible to build a fast application in Python even if it requires a lot of computing power.
Almost everything that can be done in MATLAB can also be done in Python. NumPy is a free module for matrix calculation. SciPy is a free scientific module for advanced mathematics and engineering. Those two are the base for many other modules available. I highly recommend Anaconda if you want to develop your 3D application in Python. It is a scientific-oriented working environment and it will install all the important modules for you. It will also provide access to Anaconda Cloud, a community exchange that gives you access to even more modules.
Tip: there are often more examples coming from the MATLAB community, so a good trick is to search for a solution done in MATLAB and then create the equivalent in Python.
In terms of speed of implementation, Python code is typically 3 to 5 times shorter than equivalent Java code and 5 to 10 times shorter than equivalent C++ code.
Pros:
Cons:
Honorific Mention: Julia and GNU Octave
There are many other options besides Python and MATLAB to program 3D applications. Julia is a good language for mathematics, comparable with Python, and GNU Octave is an open source software comparable to MATLAB. The main reason why I recommend Python and MATLAB over the others is because of their popularity and the number of free resources available. This is from a strictly commercial point of view, where time is extremely valuable.
A Good Object-Oriented Programming Language (C++, Java)
There are situations where this is the only option. If something can be done in Python with good results, it should be done in Python, because it will be a lot smoother and faster to develop. Sometimes it is simply not possible, because performance is too important or because it must run on an embed or minimal hardware device.
If that is the case, you should first look at the available examples in that language and pick one that offers you as many shortcuts and examples as possible for what you want to achieve.
The Fundamental Building Blocks
Once you have decided in which environment you want to develop your application, it is time to determine the building blocks of your 3D application.
Data Formats
Since the goal is to extract information from a 3D file, a data file must be imported. If your data is a mesh (.stl or .ply), the first question you should ask yourself is if you need the entire information from the mesh or if you can live with only the vertices (a point cloud). The main reason is that it is more efficient and easier to work with a point cloud, but you lose some information, such as the orientation of the surface. In most cases, that information can be discarded because you will "know" that information already.
Data Alignment in the 3D Coordinate System
Is your 3D data already aligned in a specific way, so that you can entirely skip the alignment process? This is an important question because data alignment is a essential step for most 3D analysis programs. Before any measurements can be taken, the location and orientation of the scanned object must be known, and it must be repeatable with high accuracy and stability.
There are many strategies to automate alignments. It can be done primarily by the 3D scanner operator, which is a smart and stable approach (unless the operator messes up). Using a jig and saving a coordinate system of the working environment is possible with most scanners. We still recommend to add safety alignments steps to ensure that the imported data is properly aligned, or to refine it.
The alignment can be done by fitting geometries to lock the 6 transform dimensions (3 translations and 3 rotations). Another smart approach is to use a previous scan or a 3D model, already in the desired position and perform a best-fit of the 2 shapes. You must work with what you know and figure out the safest approach to properly locate the object. It is often necessary to perform 2 alignments: the first one is the safe, smart approach that works 100% of the time. The second alignment is a refined alignment of the first. It is the final position, necessary to get accurate measurements.
3D Measurements
This may seem like the most complicated part of the application, but if you can test your methods in a 3D software with great visualization, it will be a lot smoother than expected. This is the key to get things done fast: you really need to know exactly what is happening at each step in the measurement sequence, to quickly improve or repair any broken code or find a different strategy to extract measurements.
A few strategies for geometric measurements:
- Fit 3D geometries (planes, cylinders, cones, vectors) from a point cloud or a mesh.
- Construct new geometries from the already fitted geometries (by intersecting them or by extracting their length or orientation).
- Isolate a portion of the 3D data and look at extremums.
- Slice the object and extract 2D geometries from a 2D section.
- Change the coordinate system (switching from a Cartesian to a cylindrical or spherical coordinate system can transform a complicated problem into a simple one).
Securities and Notifications
How important is it for you to know when something is not working as expected? Adding a validation function at each step in the process to test the hypotheses is strongly recommended. The operator running the application must be alerted if the application doesn’t work as expected. Examples of validation steps to consider:
- Is the object correctly aligned?
- Test the scanned data to make sure there is no missing area.
- Is the resolution of the 3D file expected (to detect a mistake from the scanner operator)?
- Detect if there is another object present in the 3D file.
- Is that measurement within the expected value range?
Even if you only extract a few measurements from a simple shape, if there are no securities to validate the hypotheses, a bug or a flaw in the process could be detected too late. A good visual (a snapshot taken in a 3D viewer) of the measurement is another added security, since the operator could see a problem that didn’t trigger an event.
Exporting and Reporting
Do you need a well-formatted report or just a table? Define the needs and they could influence your choice of solution. Polyworks comes with its own report editor. It is quite easy to save a layout and fill it automatically, which makes Polyworks a good solution to generate professional reports. On the other hand, there are many Python modules for editing in Excel, such as OpenPyXL and XlsxWriter, just to name a few. It is possible to save an Excel file and use it as a layout, and you can format your report to your liking without any programming knowledge.
Conclusion
You probably noticed that this guide is colored with my personal experience and opinion. There are of course other opinions and we can't always agree, so take these final words as my personal opinion.
Our usual customer uses the acronym ASAP at every meeting and wants to be the reference in their market when they create a new solution for an existing problem. They also want a turn-key solution because they can't afford to hire someone with "3D automation skills" just to operate the workstation or make adjustments.
For these reasons, Polyworks did it for us most of the time. When it isn’t a good option, Python is usually our second best pick. With today’s competitive markets, the professionalism and stylish appearance of software solutions is important, and we have moved a bit away from MATLAB for that reason.