Dexta Valley Coach: Let's Touch VR World's Technology, It's Realized This Way |

Gesture interaction is becoming a hot topic in the field of VR interaction. There are well-known Leap Motion and uSens abroad, and domestic teams such as Micro Motion and HandCV. It may seem natural to operate directly in the virtual world with your hands. However, when you actually experience it, you will find that you clearly see your hand grabbing something, but it feels like you have caught a bunch of air.

It is for this reason that you feel the virtual world while interacting with your gestures and become an effort. One of the most famous startup companies is Dexta Robotics. Dexta Robotics is a company dedicated to inventing the next generation of VR/MR interactive devices. Their main product, Dexmo, is a wearable hand exoskeleton that helps users to sense the volume, shape, stiffness, and force of virtual objects in virtual reality interactions.

In this issue, we invited the founder and CEO of Dexta, Aler Gu. Gu Zhi Chi is a fanatic robot enthusiast who studied mechanical control engineering at Cambridge University. After reading this book, he took a break from school.

Founder and CEO of Dexta Robotics

The following is the finishing content of this open class:

Lei Feng Network: Dexmo this product was developed since 2013, how did you think of developing such a product? Why are the difficulties that need to be overcome during such a long time?

Gu Zhichi : The main function of early Dexmo was to detect hand movements. At the time, several scenarios were conceived: virtual reality, remote robotic detonation with the control of our old, smart robots, and immersion and more precise control of telepresence. Later, with the rise of virtual reality in 2014, I noticed that more and more people are focusing on head-mounted displays. So we take a long-term view: what kind of technology will be needed by the market in 2 years?

(Lei Fengwang Note: "telepresence", TelePresence, is a new technology that creates a unique "face-to-face" experience on the Web by combining high-definition video, audio, and interactive components.)

The conclusion is interaction. After interviewing a lot of friends in the circle, we learned that bringing tactile feelings into virtual reality is a huge demand, but because of difficulties in implementation, it is clear that someone really solved it. Most of the solutions already available on the market are outdated and clearly cannot keep up with this wave of virtual reality. It is also the kind of self-confidence that is unique to technology practitioners. We believe that we can take down this tough bone with our own team ability. Then we tried to install different force feedback schemes based on existing motion detection equipment. Two years later, it turns out that we did it.

Dexmo's earliest product concept design drawing

difficulty:

The whole process of designing a device is a lot harder, because no one is ready-made. Whether it is hardware or software, it is necessary to make wheels from scratch, because the wheels that can be found on the market are not good enough. For example, the structural design of the thumb motion detection module, the optimization of the two-way communication rate of the device and the host device, the compact design of the force feedback module, the interactive logic of the software-side hand grabbing, and the like.

Slightly detailed, the biggest challenge in terms of hardware is that the entire force feedback unit is very compact with guaranteed functionality. First, the housing, gears, motor drive circuits, and embedded programs all start from zero. Dimensions, weight, cost, torque, and shape all need to be balanced. There are too many aspects involved in the optimization of the Cost function, making trade-offs more difficult. Some of the difficulties are not technical. Hardware companies have experienced that in small batch production, it is difficult for manufacturers to assist you fully because of the limitations of order quantity. I am very grateful to Mr. Chen Weijie of the Dongguan Weichuang model for their great help. They gave us a lot of selfless help in the design proofing of the Dexmo force feedback servo.

The main difficulty in software is that many things cannot be debugged without proper hardware. When there is no hardware to test, many problems will not appear at all, which indirectly lengthens the front. Many of the software engineer's work depends on the degree of stability of the hardware device. This can even be regarded as a problem of management and coordination. For example, our 3D item interaction engine transfers an object from the left hand to the right hand. This seemingly simple action actually designs to multi-point collision and alignment of the hand. Once the physical effects are involved, the complexity of its interaction also rises linearly. However, before the hardware equipment works stably, our software engineers cannot participate in debugging, so even if it is difficult to know this piece, it can only be placed first and piled into the technology pipeline.

Lei Feng network: How does Dexmo let the user experience the virtual world?

Gu Rong Chi: A simple description of the operating mechanism: Dexmo itself has 2 systems, motion detection, and force feedback system. The difficulty is how to coordinate these two parts so that they can work seamlessly. The motion detection system detects the human body's 11 degrees of freedom of hand movements. The thumb has 3 degrees of freedom, and the remaining 4 fingers each have 2 degrees of freedom. After this information is collected, it is transmitted back to the computer through the NRF.

Dexmo system simple illustration

In the PC segment we map the user's hand model to the hands of the virtual character and begin collision detection and physical state determination of the object interaction. After calculating the vector direction and magnitude of the force that should actually be generated, we send these data back to Dexmo and pass the master to each of the five force feedback modules for force simulation. When a person is grabbing an object, his hand cannot pass through the object, and the shape of the hand will conform to the shape of the body of the body, and the echo of the virtual visual content will create the illusion that the person will grasp the object.

Lei Feng network: "Dexmo allows users to feel the size, shape and hardness of the virtual object", where only the hardness is the force feedback, size and shape is how to do it? The weight of the object, the texture of the surface, these are also very important touch, can you simulate?

Gu Chi Chi: The hardness is achieved by controlling the size of the force and the output interval. The size is simulated by the geometry of the exoskeletons and the shape of the 3d virtual content. For example, if a hand is grabbing a virtual jar, the movement of the finger will be confined to the surface of the can, just as if we were grabbing a jar in the real world. The force feedback provided by several contacts at the tip of the finger let us feel the shape and size under the visual feedback and coordination of the brain.

The equipment that can generate weight in the 1990s, and I personally think that it is difficult for such products to become consumer goods that enter millions of households.

Weight is a very interesting question. Middle school physics tells us that force = reaction force. To simulate real weight, you may need to attach a force feedback robot fixed to the wall. This approach is not acceptable to consumers, so simulating weight is still a problem. We may only be able to simulate weight by creating illusion.

Dexmo tried force feedback robot design

We have had similar attempts before, but we stopped the development of the force feedback manipulator because it severely limited the user's range of motion.

Touch is another issue . Due to space limitations, I only say one thing: Many times, technology itself is not developed, but it is currently subject to cost and size restrictions. Even if it is made into a product, the high cost will make consumers Deterred.

Lei Feng Network (search "Lei Feng Net" public concern) : Force feedback on how to achieve the machinery? What kind of controller is used? How to solve the mechanical delay?

Gu Zhichi: Force feedback is not a mechanical issue but a control issue.

Simply speaking, our motor control algorithm can make the motor behave more like a spring with variable hardness, and the hardness and response interval can be very quickly adjusted by the upper computer's instructions. The controller is a microprocessor that is readily available everywhere, with the lowest cost according to the requirements. This is nothing too much to say.

Mechanical delay is a problem , but at present, our practical solution is that the motor directly drives through gears, so the response speed is already the response speed of the motor itself under the premise of gear quality clearance.

Lei Feng network: how to solve the problem of the finger through the object? This will appear on gesture interaction programs such as Leap Motion.

Gu Chi Chi: I describe it from two aspects : real physics and images.

The limitation of the physical effect comes from the reverse torque provided by the force feedback module, and the virtual torque is triggered by the virtual content. There are two conditions that need to be met here: triggering of virtual content and physical hardware to generate this force. The collision detector in the virtual world understands the collision and triggers the action of the force feedback module. This specific mechanism can actually refer to Q2.

In terms of images , we have our own set of interaction engines to limit the interaction of virtual 3D objects and hands. The combination of the two can really improve immersion.

Leap Motion's latest interactive engine actually tries to solve part of this penetration problem. They only limited the image, that is, when the hand grasps the object, it bends to a certain point, the image limits the movement of the hand, and the model also stops following the hand movement. However, in the absence of any hardware, the real-world manpower is not controlled. This is also a common problem in other hand motion capture programs.

Here I would like to point out that the vibration motor and force feedback cannot be equal.

The vibration motor is technically very easy to implement. This is also a point where a large number of “interactive gloves” entrepreneurs like to use it to fool the public and investors.....combine force feedback, tactile feedback, and weight feedback. Then simulate the two can represent the vibration motor can provide some of these feedback. This is inaccurate.

Lei Fengwang: What is Dexmo's hand motion capture technology and how accurate and delayed is it? How to cooperate with the spatial positioning system?

Gu Rong Chi: The angle sensor is used, and the theoretical accuracy index is 0.06 degrees. However, as a wearable device, the theoretical accuracy is of no reference value because the wearing error may have exceeded 0.5 degrees. In general, the interaction of virtual reality requires more follow-up than absolute accuracy. The so-called degree of follow-up, such as our device can check the 3 degrees of freedom of the thumb movement, the human in the real-world action will be a lossless map to the virtual world; compared to Oculus Touch based on capacitive sensing (capacitance sensing) finger movement guess, Or some other low-end glove program, Dexmo will give people a higher sense of control over the virtual character.

Dexmo's delay is 25ms , but the overall delay experienced by the user is not simply added to the helmet's delay, but rather the whole pipeline's operation is considered. For example, the force feedback command from the PC is displayed when the display segment is rendered. Send to Dexmo, so that part of the delay occurs in the same period.

Touch's hand motion detection is switch type, so give people control over their virtual characters is not high, many motion capture gloves based on the bending sensor is not very hard on the thumb, sometimes with a degree of freedom to send the . Of course, many glove solutions have stagnated after the input. Using the thumb and forefinger as a trigger on the Vive controller naturally saves a lot of things... but the problem is that it's useless. In short, followability is more important than the so-called accuracy.

Space tracking is already a solved problem . It is not worth spending our own engineering power to do it all over again. We have already gone to Valve's headquarters to discuss with their engineers about the integration of Lighthouse's tracker into Dexmo. There is no technical difficulty. Of course there are many other options for space tracking. We maintain an open attitude toward this.

Lei Fengwang: How does force feedback combine with gesture interaction?

Gu Rong Chi: Gesture interaction itself is a relatively natural way of interaction. For example, Microsoft's Hololens, called out the bloom gesture used in the menu, determined the tapping gesture, and so on. On this basis, by adding force feedback to the physical angle limit, the complexity of the interaction can be further improved. For example in VR we can design knobs, switches, etc. on the person's forearm and chest. Prior to this, these hardware switches can only be placed on the controller itself, and with force feedback many hardware switches can be implemented through software, which is a big improvement.

It's like having a touch screen and vibration feedback. We slowly get out of physical buttons. With force feedback, we can generate numerous buttons, knobs, switches, etc. in the virtual world. It is very possible that we will no longer need it in the future. Multi-body button.

This video contains some of Dexmo's innovations in interaction.

Lei Feng Network: Why NRF as a wireless transmission solution?

Gu Rong Chi: Because Dexmo as a two-way interactive device, both to provide input, but also produce output, the higher the data bandwidth requirements. Many off-the-shelf hardware and communication protocols do not meet our requirements for communication speed and stability. In this comparison, Bluetooth and WiFi are no longer considered, and only NRF can be selected. Of course, we also tried Bluetooth and WiFi in the early days, but it was not ideal.

Lei Feng network: Introduce Dexmo SDK, how does the developer develop and support for it, how to attract more developers?

Gu Zhi Chi: Development for Dexmo is very easy. We have done a set of plugins similar to Vive. Just use our SDK, open UNITY and you can see your hands with your head. And hand interaction, collision detection, and force generation are handled by our algorithm for developers. The developer just needs to set the collision detector and hardness for the object.

To attract developers, our current approach is to actively seek cooperation with foreign virtual reality first-line vendors. We believe that this technology itself has great potential value. But only by putting it in the right ecosystem can it maximize its value. Once there is a giant endorsement, developers will be more willing to use more advanced hardware.

Lei Fengwang: In addition to VR/AR/MR, what are the usage scenarios of Dexmo?

Gu Zhichi: Robot control is definitely a practical need. For the EOD robot, the force feedback and blindness of the hand are exactly 2 levels of accuracy. We have received some invitations from NASA and the U.S. military in the past research and development, hoping to use this technology in aerospace. Space robotic arm control, rover capture, etc., these interactions have a qualitative improvement in force feedback.

In addition to VR/MR games, many vertical fields have many application scenarios such as drawing, education, training, and so on. To dismantle aircraft engines, this kind of high-precision work does not mean that every apprentice has the opportunity to directly operate the real machine. Here we can use VR instead. Also if you just use the Vive controller to move, the immersion is missing. If you can use Dexmo, it will be more true.

Audience: Dexta crowdfunding in 2013 was later withdrawn because the project was premature?

Gu Rong Chi: (Cite Road to VR for his interview)

What we were showing at Kickstarter was a concept. When we saw people's reactions, we felt that we were being held too high. We realize that they expect to achieve more than we can do.

What most proponents need is that a final solution can be used right out of the box, and all platforms support it. The VR hardware at the time was not as it is now. We understand that if users are unable to set up a tracking system independently, they will probably not use our products. Therefore, to achieve people's expectations, research and production of products, we need much more than 2 million US dollars of funds.

At that time we faced the choice: either we accept the money, pretend that none of these problems exist, don't tell our supporters what the actual situation is, and finally come up with a garbage product that nobody would be satisfied with, or we support the supporters Responsible, confess everything to them, and then go back and continue to do research and development.

As an engineer, I don't speak much. I think the product can prove everything. I hope people will feel all the energy and effort that we have condensed in this product. We had a hard time making choices, but I'm glad that we made the right decision. And now, our products can finally reach all my expectations of it in the past.

Audience: How much is the price?

Gu Chi Chi: Will be lower than the mainstream, the specific price is currently no reference significance. After further entering production, depending on the quantity, it is possible to carry out further literary work with suppliers, and at the same time there are many production optimizations that can be done.

Pull Wire Encoder

Displacement sensor, also known as linear sensor, is a linear device belonging to metal induction. The function of the sensor is to convert various measured physical quantities into electricity. In the production process, the measurement of displacement is generally divided into measuring the physical size and mechanical displacement. According to the different forms of the measured variable, the displacement sensor can be divided into two types: analog and digital. The analog type can be divided into two types: physical property type and structural type. Commonly used displacement sensors are mostly analog structures, including potentiometer-type displacement sensors, inductive displacement sensors, self-aligning machines, capacitive displacement sensors, eddy current displacement sensors, Hall-type displacement sensors, etc. An important advantage of the digital displacement sensor is that it is convenient to send the signal directly into the computer system. This kind of sensor is developing rapidly, and its application is increasingly widespread.

Magnetic Scale Linear Encoder,Magnetic Scale Encoder,Encoder Software,Encoder Meaning

Changchun Guangxing Sensing Technology Co.LTD , https://www.gx-encoder.com