Notes On the New Mac Background Replacement Feature

Gary from MacMost.here. Let's explore the webcam background replacements feature in macOS Sequoia. 

So a new feature in macOS Sequoia is when using your Mac's webcam you can replace the background behind you with a gradient or another image. Apple actually shows this in the context of using FaceTime or some other video conferencing app. But you can actually use it to record video as well. 

Let's start with the FaceTime App and look at the basics. Here I'm using FaceTime and what you're actually seeing back here is my actual wall. I've got a green screen there and I've got a green cover on my chair. But this will work with anything behind you. I'll show you that in a minute. So to turn this On you want to look in your Mac's Menu Bar for this little icon here showing a screen with a person. Click on that and this gives you all of the options for adjusting what your webcam is doing. You've got lots of other things like using Portrait Mode, Studio Light, and all of that. But the feature you want to look for is Background. You turn that On it will replace the background behind you with what is shown here. In this case a blue gradient. You can change the gradient by clicking here and choose another one. Notice there are three tabs here. So you've got your gradient colors but you've also got the ability to choose from these images that Apple provides. So you can pick one of these and it can look like you're at Apple's Headquarters, or some other place around there. 

You can also click here and then you can choose something from your Photos here and choose something from your Photos Library or you can choose a File. So I'm going to choose this one right here and it will bring that in. Then you can see it uses that behind you now. Now like I said you don't need a green screen. So here's me actually sitting with my Mac Book with my bookcases behind me and you can see there's all sorts of things, even moving things there on the bookcases. I can use the same feature there. So it's basically removing the static background behind you using Machine Learning to figure out what is you there in the foreground as opposed to what is behind you in the background. 

Now there do seem to be some requirements for this even though Apple's macOS Sequoia page doesn't show any. It shows that it should work for anybody with Sequoia. But in practice I find this doesn't work at all with third party cameras. It only works with the FaceTime cameras that are built-in to all MacBook models and iMac Models. So make sure you're using your MacBook or iMac's camera. I find it also works with the Apple Studio Display's camera. But I haven't seen it work with third party cameras or any kind of other video feed coming in through USB or anything like that. I have also heard from a lot of people that say it doesn't seem to work with the few Intel Macs that still will work with macOS Sequoia.

It does, however, work if you use your iPhone as a webcam. So you can connect your iPhone using Continuity Camera and use your iPhone as a webcam on a Mac Mini or a Mac Studio and the features are available there. 

By the way, if you find these videos useful you can join the more than 2000 others that support MacMost at Patreon. You get exclusive content, course discounts, and more. You can read about it right here. 

Now you can use this feature in other video conferencing apps. But some apps, like Zoom, actually have this feature built-in so you're probably better off using Zoom's Background Replacement when using Zoom than using this. However, you can also use it for other apps that you use the webcam. For instance the most basic one is to use QuickTime Player. So let's launch that and in QuickTime Player you can use File and then New Movie Recording and that brings up the Movie Recording Window. You can choose the Camera. So in this case I've chosen my Studio Display Camera here and you can see the background has been replaced there. It still remembers the setting from before. So I can now choose another background here. Let's go in and choose a different color or I can choose this image here and it works in QuickTime Player. If I were to record this and record the video and stop you can see it actually recorded it with a background replacement. So this could be really useful for recording things for presentations, or making your own videos. You don't have to worry about what's behind you on the wall. You can simply use this feature here, record things in QuickTime Player and then bring them into iMovie or Keynote and use them there. 

You can also choose some of these other options here like, for instance, Center Stage will align you there, Portrait Mode will blur things in the background if you don't want to use this, and Studio Lighting will change the lighting around you. You can see there are options for all of these here which can make things really nice and allow you to create some really nice just video capture of you. 

But here's something else you can do. You can take an image that's just green and bring that in as Custom Image. So let's go and do that here. I'm going to change the background and I'm going to bring this in. I can drag and drop it in and switch to it. Now you can see it has given me a virtual green screen. I can have all sorts of stuff, be outdoors or whatever, and it's recording me as if I am using a green screen. So then when I go to use this video here I've got a green screen background that can easily remove in iMovie to make the background completely transparent. 

Now if your goal is to use this in iMovie you can skip QuickTime Player and just record directly in iMovie. iMovie has the ability to use your Mac's webcam and record things. So, you move to File and then Import Media. Then the first option here should be the webcam built into your MacBook or your iMac or a Studio Display. It doesn't seem to work with any third party cameras and doesn't seem to work with the iPhone as well, interestingly enough. So, you can select this here and you can see in record this is my regular green screen behind me. But notice if I go in and turn Backgrounds on it does use these backgrounds. So I can record with a gradient there or I can go and it's going to be hidden behind here but choose one of these other gradients or picture or one of these custom pictures including the green screen. So I can record this and it will record me with the fake green screen no matter what's behind me. Then when I'm done I have that video here in iMovie and I can drop it on top as a cutaway. With cutaways you can set them to green screen, of course. You can see it actually works. It will use this as a green screen. iMovie doesn't know that it's a fake green screen. It thinks it is a real one. It works just fine. 

Now note that iMovie does have the ability to move you around when you're in a green screen. You're simply going to cover the whole frame. So if you do want to use like Picture-in-Picture and put yourself in the corner there it won't let you remove the background. So if that is your goal you might be better off actually recording with the gradient or some other background there and then put yourself in the corner. It's the way to get the stuff behind you out of your video and have it be something more neutral even though it's not transparent. 

Now those aren't the only apps that can use this. For instance the Photo Booth App, something people usually forget is on their Macs, also works with this. So you can use Photo Booth here and notice that if I turn Backgrounds on it uses these backgrounds here. So you can record by switching to Video here and record in Photo Booth and it will start recording you and then when you're done you've got the video here, which you can either Control Click and Export, or just Drag out. So just an alternative there. 

You also have the ability in Photo Booth to do something else which is take a picture. It will use these backgrounds when it takes the picture. So, for instance, let's switch over to the image here like this and I can have it take a picture of me in Photo Booth and there's the picture. I can drag that out here and that's just a regular JPEG file now, but with the background replaced. 

What about Keynote? Well if you want to put video on a frame here in Keynote you would normally record that in something like QuickTime Player and then drag and drop it in. Then you can have that video play. So, of course, you can use this feature when recording in Keynote. But you also have the ability to insert Live Video. So a live feed here and you definitely can go in here and use this feature in Keynote. So you can have these backgrounds be something else. But it should be noted that Keynote kind of already has this functionality with Live Video. If you go to Live Video here you had the ability to turn on the background feature and you could Fill with a color or gradient or something else. Even an image. So you already kind of had this going on in Keynote as a Keynote specific feature. But you certainly could record something in QuickTime Player and use that in Keynote if that's an easier process for you. After all the live video feature here is meant for recording your full presentation and then it will playback that live video or for presenting live. When presenting live it is interesting to actually be able to remove the background behind you especially if it is some ugly wall or something like that while you're standing in front of people presenting.

In general this works with all third party apps that I tested as well as long as you're using the built-in camera in your Mac or Apple Studio Display it seems to change the video feed before the app gets the video. In third party apps it is particularly useful because sometimes you do want to use green screen with those third party apps and you don't have a green screen available. So this way you can fake it using the background replacement feature. 

So as an example let me switch over to doing that right here. So now this is my Apple Studio Display camera using the background replacements. It doesn't matter what is behind me. If it's a green screen, if it's bookcases, if it's a window looking outdoors I can use this fake green screen process and actually record my videos this way. Now the Apple Studio Display camera isn't as high resolution as my regular camera. But that probably wouldn't be as noticeable if I just did the entire video this way. 

So this is a great tool that goes beyond just video conferencing because I think it will help us make better videos for iMovie and for our presentations as well. Hope you found this useful. Thanks for watching.