Video Transcript
REtracker FIZZ/BLISS Tutorial (Part 2)
Marwan Rassi · 2,900 views 26 Jul 2023 · Duration 16:12 · youtu.be/lKOrcU9PKxk
Creating an Unreal Engine 5 project from scratch to be ready for REtracker Fizz and REtracker Bliss Credit: K. Rassi (carl.rassi15@gmail.com)
Auto-generated YouTube captions, grouped into ~45-second blocks. Product and technical terms may be misspelled. Timestamps link to the video.
[Music] hello everyone thank you for joining us for part two of this demo video in this session we will delve deeper into the setup process of the retracker Fizz and Bliss with Unreal Engine together we will explore the steps required to seamlessly integrate these components into your Unreal Engine projects so let's get started launch your Unreal Engine editor as you can see we are using version 5.2 for this demo first create a blank project and give it a name [Music]
next navigate to the plugins and search for live link enable the following plugins live link live link camera live link free d and live link lens also remember to activate camera calibration and lastly the timed data monitor restart the Unreal Engine editor for the assigned plugins to take effect properly we are going to create blueprints for Blackmagic time step and Blackmagic time code so create a blueprint folder and place them inside right-click your mouse
and go to the blueprint class expand all classes and search for time step select Blackmagic custom time step and name it whatever you like next right-click your mouse again go back to the blueprint class and search for time code select Blackmagic time code provider and name it as per your preference double-click the Blackmagic time code you just created and set the time code configuration settings based on your video format and frame rate compile and save it repeat the same process for the Blackmagic time step you created earlier
navigate to the project settings and search for time step set your custom time step to the one you created earlier similarly search for the time code in the project settings and assign the time code provider you previously created ensure to set the frame rate that matches your video's frame rate create a folder for the media bundle then go to the media profile and click on new empty media profile afterward click on media profile and click select now choose the folder you just created and give the media profile a name of your preference click save and proceed the editor will likely prompt you with the project doesn't have the media proxies configured so make sure to proceed with configure now
next set your preferred number of source proxies for your video input and output select the same content folder you created earlier and then click on configure this will generate a few media bundles and media sources double-click on the New Media profile you just created under media sources choose your video capture card for First Media source be sure to set the correct video format and frame rate [Music] repeat the same process for media output [Music] foreign [Music] once you have finished with the settings please save and apply them now drag the First Media bundle into your scene to check the incoming SDI
video feed from your camera please check the timed data monitor ensure that your scene is synchronized with the incoming SDI video feed when you are done with the verification you can hide it somewhere in your scene before we continue any further we extend our heartfelt thanks to Greg Corson for his contribution in developing the retractor Bliss live link plugin this awesome plugin significantly speed up the virtual production lens calibration process making it incredibly easy to achieve perfect nodal offset and precise alignment between your media plate and virtual environment for those interested in learning more about virtual production we encourage you to explore Greg corson's YouTube channel where you can find valuable insights and demonstrations to get the plugin simply access the download link provided in the video
description below download the preferred version and unzip the downloaded file copy the plugins and tag to print folders navigate back to your unreal project folder and paste them there now return to the downloaded folder and locate the content directory copy April tag arukotag and VP Studio core into the content directory of your unreal project inside the tag to print folder you will find the April tag arukotag and checkerboard print the April tag ID number one and checkerboard on A3 size paper for the aruko tag use its original size and print it on A4 size paper now return to
the unreal editor where you will notice a few newly added folders in your content directory drag the April tag into your scene and repeat the same for the aruko tag under the virtual production category add a checkerboard to your scene set the number of corner rows and Corner columns for the checkerboard when counting the corner points on the checkerboard make sure to count only the intersections where the black and white squares meet these intersections are the corner points for this case it is four corner rows and six Corner columns don't forget to measure the square size of your printed checkerboard in this case it is 5.1 centimeters this measurement is crucial for achieving accurate camera calibration I will use red for the odd and black for
the even [Music] next select the April tag and set its size according to the measurement on your printed April tag in my case it is 23.9 centimeters verify the size setting for the aruko tag as well and it should be set to 15 centimeters to ensure that none of the tags in your scene are being blocked I'll lower the floor slightly once the adjustment is made remember to save your progress and restart the unreal editor this step is necessary to load the retract or Bliss live link plugins we copied earlier Now navigate to the plugins section and enable live link Bliss restart the unreal editor once again to allow the plugin to take effect
[Music] go to the live link Tab and click on Source you will find live link Bliss source with available options select it and click on ADD with the default settings currently no data is being received to enable tracking data from the Bliss sensor certain settings need adjustment launch the retractor Bliss settings and ensure that both live link Bliss and tag tracking options are enabled enabling these options will allow the live link Bliss plugin to receive tracking data from your bliss sensor leave the free d option unchecked since it's not used in this case adjust the tag marker size in centimeters to match the measurement on the printed April tag similarly modify the marker size under world pose options accordingly let the rest of the settings remain at their default values save the changes and proceed to launch retract or Bliss main
application [Music] return to the unreal editor and you will notice that the status of the tracking data has turned green next select live link 3D and update the IP address and port number settings these settings are required for receiving tracking data from retracker Fizz lens data interface in the previous tutorial we assigned the IP address with port number 38800 to this machine therefore proceed by setting the IP address and the port number accordingly thank you live link Bliss is receiving a total of two sets of tracking data the first set labeled as camera one will be used for camera tracking while the second set
with the subject name camera 101 will be used for tracking the April tag during the nodal offset process on the other hand live link 3D is receiving only one set of tracking data with the subject name camera 250. this lens data will be utilized for the lens mapping process later on for the lens mapping process we will need a lens file let's start by creating a folder for it and placing the file inside [Music] right-click using the mouse then navigate to miscellaneous and select lens file proceed by giving it any name you prefer [Music] next add a blank actor to your scene and remember to zero out its location rename this blank actor as camera parent since
it will be used as the parent of your cine camera later on after that add a cine camera actor foreign ER drag and place it under the blank actor as a child while the cine camera actor is still selected navigate to the details tab click on ADD and then search for live link select live link controller and proceed by providing a name of your choice to indicate that this controller will be used for camera tracking you can add Bliss at the end of the name next add another live link controller for lens data and you can add Fizz at the end of the name lastly click on ADD and search for lens choose lens from the options and proceed to change its name
if necessary now proceed by selecting the live link controller for Bliss that you previously created assign camera one for the subject representation then navigate further down to the camera roll and set it to none as we won't be using the lens data from this subject next choose the live link controller for Fizz for the subject representation select camera 250 set the transform role to none and leave the camera roll unchanged as we will only utilize its lens data continue down to the lens file section and assign the lens file you previously created finally select the lens controller and assign the same lens file to it ensure that the apply Distortion option is checked and enabled additionally in the film back override section change it
to lens file locate the April tag in the outliner and proceed to the details tab select scene and then click on ADD search for live link and choose live link controller proceed by providing a name of your choice for the controller choose camera 101 as the subject representation observe how the location and rotation values are updating indicating that the April tag in your scene is being tracked by the printed tag ensure to save your live link sources as a preset so that you can load it again in the future double-click on the lens file to verify if the raw Fizz input is receiving any lens data use the nucleus M hand unit to turn the focus wheel to its minimum and maximum positions repeat this process several times to ensure that both extreme points
are reached by performing these actions the raw Fizz input will detect the range of data from the retractor Fizz and normalize it the values will be scaled to zero for the minimum and one for the maximum next let's look at the raw Iris data now before we can receive the data for the iris position we need to enable it under user-defined encoder data Focus distance and focal length encoder data are turned on by default as I am turning the iris wheel the raw Iris data is updating lastly the raw Zoom data please repeat the same process We performed earlier for focus and Iris data this will allow us to obtain the minimum and maximum values for the raw Zoom data and ensure
its accurate calibration next we will proceed to create a lens mapping curve for Focus Iris and zoom let's start with Focus first ensure that the focus is at its minimum position then click on the plus sign to add the minimum Focus key Point enable the input focus and input your minimum Focus distance referring to the marker on your focus ring for my case I will use 95 CM as my minimum Focus distance next smoothly turn the focus wheel until it reaches its maximum position and then click on the plus sign to add the maximum Focus key Point again enable the input focus and input your maximum Focus distance referring to the marker on your focus ring for my case the value is approximately
two thousand centimeter representing my Infinity Focus distance the same process to create a lens mapping curve for the iris [Music] before proceeding with focal length we need to provide some lens information for the lens file for the Lens model I will input Sony 18 to 110 millimeters the serial number can be any random combination of numbers as for the sensor Dimension I found that it is 23.6 by 13.3 for my Sony fs7 camera regarding the image resolution I will set it to
1920 by 1080 pixels for my case repeat the same process to create a lens mapping curve for the zoom this mapping curve is for testing purposes and its aim is to verify that the zoom data from retract or Fizz is functioning correctly please take note that the focal length mapping Curve will be deleted and replaced during the lens Distortion calibration process [Music] foreign [Music]
as I turn the iris wheel I notice that it doesn't really change the iris value for myceny camera however we can make it happen by adding a post-process volume set the metering mode to manual and adjust the exposure compensation to around 9.5 with these adjustments the iris Now updates appropriately as I turn the iris wheel providing the desired control and responsiveness that concludes this section of the tutorial in the next video I will guide you through the lens calibration process along with the nodal offset and virtual set alignment with your media plate thank you for watching and stay tuned for more
Transcribed 20 blocks · Source: YouTube