Video Transcript
Perfect Nodal Offset for Virtual Production using RETracker Bliss. No More Sliding Feet!
Greg Corson · 8,599 views 23 Nov 2022 #unrealengine #virtualproduction #RETrackerBliss · Duration 16:18 · youtu.be/ChzEXe30MIk
Sliding feet and misalignment are a common problem for Virtual Production. RETracker Bliss can measure your lens nodal point and tracking offset to fix this for you. This works similar to the VIVE method I've shown before, but now you can do it with RETracker Bliss and an Apriltag. Variations of this method can be used on any tracking system that can track another object in the scene besides the camera. This tutorial shows how. It starts assuming you have already calibrated your lens distortion, if you haven't done this please check out my earlier tutorials below. If you need more info on how to use this method with other trackers, please leave a comment and I'll try to help. If you want to
Auto-generated YouTube captions, grouped into ~45-second blocks. Product and technical terms may be misspelled. Timestamps link to the video.
hi everyone I'm Greg Corson and I know you want a perfect virtual production setup without any slipping or sliding or misalignment of objects they'll like this one but for this you need to know your lens nodal point and the offset of the tracking sensor this can be hard to do well now we have a way that's quick and automatic and easy unreal's lens calibrator can do this with any system that can track your camera and another object in the scene at the same time this means it'll work with systems like Vive optitrack Vicon and others if you have a Vive there's a link in the video description below that covers that if you have a different system leave a comment and I'll try to get to it today I'll show you how to do it with retract
or Bliss the tracker I've used for most of my recent videos if you're interested in getting a retract or Bliss just follow the link in the description below to get to their website click on retract or Bliss and then click the purchase retract or Bliss button here by the way I don't work for retracter and no money has changed hands between us they didn't sponsor this video but they did give me a retract or Bliss unit so that I could test it out and review it from now until the end of 2022 you can also get a 25 discount by clicking add promotion code here on the order page and entering the code corson25 that's Corson 25.
more details if you want them in the description below to use this method with retract or Bliss you need to get the latest version of my Livelink drivers from my GitHub site this includes the drivers some calibration charts to print and a sample project that you can use to follow along there are versions for unreal 5.0 and 5.1 just look in the video description below before you start you'll need to have a printed version of oruko number eight and April tag number one if you don't already have them these are included in the package this tutorial will show you how to make a lens file you can use in any of your other projects remember that if you have a lens file for unreal 5.0 you can use it in a 5.1 project but not the other way
around so if you're used multiple versions of unreal you might want to start by making a lens file in 5.0 if you use this in a 5.1 project it'll automatically convert the first part of this process is to use the unreal lens calibrator to measure your lens distortion I've already done a couple tutorials on this so I'm not going to bore you by repeating it here if you don't know the procedure already you can check one of the two links below for how to do it as you can see here I've already got the lens file with the calibration for my lens the Distortion parameters camera intrinsics everything but the nodal Point offset which is what I'm going to show you how to get now if you've struggled with this before you're going to love how much easier this is so
here's the setup the first thing I'll do is open up My Lens file and then I'll go back to the outliner here find the video input for the lens file and connect it up and then we'll go back to looking at the lens file again now you can see here's the view from my camera and I've got the aruko tag sitting on the floor here which is going to Mark the center of my studio then if I tilt the camera up you can see the April tag number one sitting on this chair now I have to set up retract your bliss so that it'll track both the camera and the tag I'm
going to use their nice settings program here to do it remember that although I'm using retracker Bliss you can do this with other types of tracking sensors you just have to be able to track two things at the same time the camera and something else so to set up retracker I'm going to enable tag tracking here by clicking the other thing is it needs to know the size of your April tag and for me it's 21.4 centimeters that's already set up and I'd recommend leaving tag marker at one and tag camera ID at 101. then I'll save it now I can just start retracker Bliss and you can see it's running 500 frames per second
the live link is connected to Port 50 000. and it's detecting tag number one if you don't see it detecting your April tag you may need to move the camera a little bit to make sure that the tag is in view when this changes to one you're all set you should have already set up your live link to use the Bliss and it should look something like this these two lines here may not appear immediately but once retracker Bliss starts they should come up and change from yellow to Green showing that they're working correctly like they are here the last bit of setup is to make sure unreal knows the size are April tag and our Roku tag and that's pretty easy and start by clicking on the April tag and you'll see in its details is a number Tag size
now my tag was 21.4 so it's already set correctly you want to measure the size of your tag as accurately as you can that'll affect the results if you don't get it right same thing for the uruko you click on that in my case the my Roku is 15 centimeters across now you're ready to go now we go back to the lens calibrator and we click on node offset and I pulled in my tag really close you can see that the two versions one from CG and one from my camera are not aligned yet that's normal I always like to start with the tag very close you want to choose noodle offset points method and you want to choose the April tag calibrator should be multiple values and
you should see it say Center here now to avoid any confusion on what you click on you can take the transparency turn it to zero so that you're only looking at the image from your camera now all we have to do is click on the points as it asks us to let's start with Center foreign top right bottom right bottom left top left you can see that as I click on them it adds them to the list over here now we need to repeat this process a couple of times to give the thing a lot of points to work with I usually try to do three up close three at medium and sometimes at least one or two in the distance you may be able to
get good results with less the most important thing is that the tag needs to be shot at different distances from the camera and not always in the center and not always straight onto the camera I'm going to do this now and I'll speed the film up so that you don't have to wait through it now you can scroll back up and set your transparency to 0.5 then scroll down and click the add to nodal offset calibration button which will add all of
these readings to your calibration you'll see a dialog like this one and the reprojection error normally seems to be zero now you can see that the CG tag has kind of jumped on top of the real one and if I move the camera around they should pretty much stay aligned you can do some further testing by moving the tag further in and further out and see if it stays aligned in all cases normally you can get an error down to almost zero or at least at half a
centimeter or so now I'm going to use the uruko tag I put on the floor to align my virtual set with my real stage I'll pan down just so that I can see it and I come up here and change the this to noodle offset a ruko marker and make sure the aruko is selected then I should be able to just click on this image and you can see it gave me four points from the uruko tag I clicked on now I just say apply to camera parent and everything jumps into line now I can go back to the main unreal screen and I can click into my comp
and blow it up a bit so you can see it you can see that this Cube which I put at zero zero zero and this axis that I aligned with the corner of the paper are all right the center of the tag is actually now the center of my set now I can move the camera around a little bit and see that everything stays in line as you can see it's holding pretty accurately it's always a good idea
before you start this process to have at least a rough idea of your nodal Point offset so that you can see if these numbers coming out here make sense normally if you get a bad result they'll be off by quite a bit and you can usually tell if the result is good or not just by panning the camera around as I did earlier if you're not getting the right results you can do some more clicks on your April tag and add those to the Noodle offset calibration to kind of refine the results and if you find it it's just kind of going completely South on you you can go back to the lens panel click on one of these nodal offset numbers and then click this trash can and it will throw away all the images that you've captured so you can start over again now for the last double check
we'll go back to the unreal scene View and we'll click on the composite and bring up a larger version of it and then bring up the calibrator again and see here that these two images aren't quite the same in the composite up here this cube is out of line by maybe half a centimeter or so you can see the red line doesn't line up with the edge of the paper like it does over here well why is this it's because of a little-known checkbox that wasn't documented very well until recently I'll show you where that is and how to fix it we'll go back
and look at the unreal again click on your CG element and then you scroll down to lens Distortion in it and you'll notice this box that says apply distortion and a distortion Source this should be the lens file you just used to calibrate your lens now watch I click on the check box then everything jumps into line What's Happening Here is I think everyone was kind of assuming that referencing the lens file down inside of your camera was enough to apply the Distortion from the lens file but apparently it's not
you need to tell the CG element to apply the Distortion and that's what that check box just did now everything should look really nice and as you can see in this full screen view we're looking pretty good I rearranged the scene a little bit so the robot would be in a good place in my studio you can see I've got him alongside this measuring scale it shows he's about 180 centimeters tall but you'll notice that as I move the camera around his feet don't slip and slide which is the most common problem you get in Virtual production that should give you a pretty good idea
how to do this and it shows you what the results should look like and I think they're pretty good it turns out there is one slight difference between unreal 5.0 and unreal 5.1 and I'll give you a quick view of what that is in unreal 5.0 in your cine camera actor you can see these live link components here and if you did the setup from my earlier tutorial you'll know that under the Fizz you scroll down a bit and you set your lens file in it well for unreal 5.1 you need one additional component inside of your cine camera actor and I'll show you that you just press the add
and type in lens and you'll see that there's a lens Distortion component all you need to do for unreal 5.1 is add one of these then go down into its details and set the Distortion source to your lens file and click apply distortion foreign just like I mentioned earlier you need to also set this up inside your CG element now I'm not in unreal 5 right now so this looks a little different but all
it'll be different is you'll still see the apply Distortion checkbox which needs to be on but the Distortion Source instead of being your lens file will be the name of that lens Distortion component that we just created that's the only difference by the way you may have noticed in the example project that the the composure setup is a little different I'm not using a green screen I'm just inserting the robot in front of the video the reason I'm doing this is because it's a little easier to set up for people in that they don't have to get their key right and you can also test to see whether the slipping is occurring without having to have a person walk into the scene which can be really handy if you're working by yourself
it also makes it very easy to tell if the robot's feet are slipping because you can see them against the actual floor of the room so if anything gets out of whack you'll notice it right away if you want a standard chroma key background with the live video feed full of people in front of it just follow any of my earlier tutorials that show how to set this up all you really need to do is reverse the order of the CG plate and the media plate in composure and that's basically it for this tutorial by now you should have a pretty good idea how you can set your nodal Point using just an April tag and the Bliss and if you don't have a bliss I've at least given you a clue how you can set up some other kind of tracking system to do the same thing
please let me know if you have any problems with this by writing in the comments and if you're trying to set up a different kind of tracking system and having trouble also let me know and I'll see if I can do a more detailed tutorial on that I hope you found this useful remember to like And subscribe and I'll see you next time
Transcribed 22 blocks · Source: YouTube