This audio is auto-generated. Please let us know if you have feedback. President Donald Trump signed an executive orderFriday that scratched or revised several of his Democratic predecessors’ major cybersecurity initiatives. “Just days before President Trump took office, the Biden Administration attempted to sneak problematic and distracting issues into cybersecurity policy,” the White House said in a fact sheet about Trump’s new directive, referring to projects that Biden launched with his Jan. 15 executive order. Trump’s new EO eliminates those projects, which would have required software vendors to prove their compliance with new federal security standards, prioritized research and testing of artificial intelligence for cyber defense and accelerated the rollout of encryption that withstands the future code-cracking powers of quantum computers. “President Trump has made it clear that this Administration will do what it takes to make America cyber secure,” the White House said in its fact sheet, “including focusing relentlessly on technical and organizational professionalism to improve the security and resilience of the nation’s information systems and networks.” Major cyber regulation shift Trump’s elimination of Biden’s software security requirements for federal contractors represents a significant government reversal on cyber regulation. Following years of major cyberattacks linked to insecure software, the Biden administration sought to use federal procurement power to improve the software industry’s practices. That effort began with Biden’s 2021 cyber order and gained strength in 2024, and then Biden officials tried to add teeth to the initiative before leaving office in January. But as it eliminated that project on Friday, the Trump administration castigated Biden’s efforts as “imposing unproven and burdensome software accounting processes that prioritized compliance checklists over genuine security investments.” Trump’s order eliminates provisions from Biden’s directive that would have required federal contractors to submit “secure software development attestations,” along with technical data to back up those attestations. Also now eradicated are provisions that would have required the Cybersecurity and Infrastructure Security Agency to verify vendors’ attestations, required the Office of the National Cyber Director to publish the results of those reviews and encouraged ONCD to refer companies whose attestations fail a review to the Justice Department “for action as appropriate.” Trump’s order leaves in place a National Institute of Standards and Technology collaboration with industry to update NIST’s Software Software Development Framework, but it eliminates parts of Biden’s order that would have incorporated those SSDF updates into security requirements for federal vendors. In a related move, Trump eliminated provisions of his predecessor’s order that would have required NIST to “issue guidance identifying minimum cybersecurity practices”and required federal contractors to follow those practices. AI security cut Trump also took an axe to Biden requirements related to AI and its ability to help repel cyberattacks. He scrapped a Biden initiative to test AI’s power to “enhance cyber defense of critical infrastructure in the energy sector,” as well as one that would have directed federal research programs to prioritize topics like the security of AI-powered coding and “methods for designing secure AI systems.” The EO also killed a provision would have required the Pentagon to “use advanced AI models for cyber defense.” On quantum computing, Trump’s directive significantly pares back Biden’s attempts to accelerate the government’s adoption of post-quantum cryptography. Biden told agencies to start using quantum-resistant encryption “as soon as practicable” and to start requiring vendors to use it when technologically possible. Trump eliminated those requirements, leaving only a Biden requirement that CISA maintain “a list of product categories in which products that support post-quantum cryptography … are widely available.” Trump also eliminated instructions for the departments of State and Commerce to encourage key foreign allies and overseas industries to adopt NIST’s PQC algorithms. The EO dropped many other provisions of Biden’s January directive, including one requiring agencies to start testing phishing-resistant authentication technologies, one requiring NIST to advise other agencies on internet routing security and one requiring agencies to use strong email encryption. Trump also cut language directing the Office of Management and Budget to advise agencies on addressing risks related to IT vendor concentration. In his January order, Biden ordered agencies to explore and encourage the use of digital identity documents to prevent fraud, including in public benefits programs. Trump eliminated those initiatives, calling them “inappropriate.”  Trump also tweaked the language of Obama-era sanctions authorities targeting people involved in cyberattacks on the U.S., specifying that the Treasury Department can only sanction foreigners for these activities. The White House said Trump’s change would prevent the power’s “misuse against domestic political opponents.” Amid the whirlwind of changes, Trump left one major Biden-era cyber program intact: a Federal Communications Commission project, modeled on the Energy Star program, that will apply government seals of approval to technology products that undergo security testing by federally accredited labs. Trump preserved the language in Biden’s order that requires companies selling internet-of-things devices to the federal government to go through the FCC program by January 2027. #trump #scraps #biden #software #security

Real-world data is often costly, messy, and limited by privacy rules. Synthetic data offers a solution—and it’s already widely used: LLMs train on AI-generated text Fraud systems simulate edge cases Vision models pretrain on fake images SDVis an open-source Python library that generates realistic tabular data using machine learning. It learns patterns from real data and creates high-quality synthetic data for safe sharing, testing, and model training. In this tutorial, we’ll use SDV to generate synthetic data step by step. pip install sdv We will first install the sdv library: from sdv.io.local import CSVHandler connector = CSVHandlerFOLDER_NAME = '.' # If the data is in the same directory data = connector.readsalesDf = dataNext, we import the necessary module and connect to our local folder containing the dataset files. This reads the CSV files from the specified folder and stores them as pandas DataFrames. In this case, we access the main dataset using data. from sdv.metadata import Metadata metadata = Metadata.load_from_jsonWe now import the metadata for our dataset. This metadata is stored in a JSON file and tells SDV how to interpret your data. It includes: The table name The primary key The data type of each columnOptional column formats like datetime patterns or ID patterns Table relationshipsHere is a sample metadata.json format: { "METADATA_SPEC_VERSION": "V1", "tables": { "your_table_name": { "primary_key": "your_primary_key_column", "columns": { "your_primary_key_column": { "sdtype": "id", "regex_format": "T{6}" }, "date_column": { "sdtype": "datetime", "datetime_format": "%d-%m-%Y" }, "category_column": { "sdtype": "categorical" }, "numeric_column": { "sdtype": "numerical" } }, "column_relationships":} } } from sdv.metadata import Metadata metadata = Metadata.detect_from_dataframesAlternatively, we can use the SDV library to automatically infer the metadata. However, the results may not always be accurate or complete, so you might need to review and update it if there are any discrepancies. from sdv.single_table import GaussianCopulaSynthesizer synthesizer = GaussianCopulaSynthesizersynthesizer.fitsynthetic_data = synthesizer.sampleWith the metadata and original dataset ready, we can now use SDV to train a model and generate synthetic data. The model learns the structure and patterns in your real dataset and uses that knowledge to create synthetic records. You can control how many rows to generate using the num_rows argument. from sdv.evaluation.single_table import evaluate_quality quality_report = evaluate_qualityThe SDV library also provides tools to evaluate the quality of your synthetic data by comparing it to the original dataset. A great place to start is by generating a quality report You can also visualize how the synthetic data compares to the real data using SDV’s built-in plotting tools. For example, import get_column_plot from sdv.evaluation.single_table to create comparison plots for specific columns: from sdv.evaluation.single_table import get_column_plot fig = get_column_plotfig.showWe can observe that the distribution of the ‘Sales’ column in the real and synthetic data is very similar. To explore further, we can use matplotlib to create more detailed comparisons—such as visualizing the average monthly sales trends across both datasets. import pandas as pd import matplotlib.pyplot as plt # Ensure 'Date' columns are datetime salesDf= pd.to_datetimesynthetic_data= pd.to_datetime# Extract 'Month' as year-month string salesDf= salesDf.dt.to_period.astypesynthetic_data= synthetic_data.dt.to_period.astype# Group by 'Month' and calculate average sales actual_avg_monthly = salesDf.groupby.mean.renamesynthetic_avg_monthly = synthetic_data.groupby.mean.rename# Merge the two series into a DataFrame avg_monthly_comparison = pd.concat.fillna# Plot plt.figure) plt.plotplt.plotplt.titleplt.xlabelplt.ylabelplt.xticksplt.gridplt.legendplt.ylim# y-axis starts at 0 plt.tight_layoutplt.showThis chart also shows that the average monthly sales in both datasets are very similar, with only minimal differences. In this tutorial, we demonstrated how to prepare your data and metadata for synthetic data generation using the SDV library. By training a model on your original dataset, SDV can create high-quality synthetic data that closely mirrors the real data’s patterns and distributions. We also explored how to evaluate and visualize the synthetic data, confirming that key metrics like sales distributions and monthly trends remain consistent. Synthetic data offers a powerful way to overcome privacy and availability challenges while enabling robust data analysis and machine learning workflows. Check out the Notebook on GitHub. All credit for this research goes to the researchers of this project. Also, feel free to follow us on Twitter and don’t forget to join our 95k+ ML SubReddit and Subscribe to our Newsletter. Arham IslamI am a Civil Engineering Graduatefrom Jamia Millia Islamia, New Delhi, and I have a keen interest in Data Science, especially Neural Networks and their application in various areas.Arham Islamhttps://www.marktechpost.com/author/arhamislam/Step-by-Step Guide to Create an AI agent with Google ADKArham Islamhttps://www.marktechpost.com/author/arhamislam/Implementing an LLM Agent with Tool Access Using MCP-UseArham Islamhttps://www.marktechpost.com/author/arhamislam/Implementing an AgentQL Model Context ProtocolServerArham Islamhttps://www.marktechpost.com/author/arhamislam/Implementing An Airbnb and Excel MCP Server #stepbystep #guide #creating #synthetic #data

On most of the projects I’ve seen, there are a lot of tasks developers go through that are repetitive and error-prone, especially when it comes to integrating new art assets. For instance, setting up a character often involves dragging and dropping many asset references, checking checkboxes, and clicking buttons: Set the rig of the model to Humanoid, disable the sRGB of the SDF texture, set the normal maps as normal maps, and the UI textures as sprites. In other words, valuable time is spent and crucial steps can still be missed.In this two-part article, I’ll walk you through hacks that can help improve this workflow so that your next project runs smoother than your last. To further illustrate this, I’ve created a simple prototype – similar to an RTS – where the units of one team automatically attack enemy buildings and other units. With each scripting hack, I’ll improve one aspect of this process, whether that be the textures or models.Here’s what the prototype looks like:The main reason developers have to set up so many small details when importing assets is simple: Unity doesn’t know how you are going to use an asset, so it can’t know what the best settings for it are. If you want to automate some of these tasks, this is the first problem that needs to be addressed.The simplest way to find out what an asset is for and how it relates to others is by sticking to a specific naming convention and folder structure, such as:Naming convention: We can append things to the name of the asset itself, therefore Shield_BC.png is the base color while Shield_N.png is the normal map.Folder structure: Knight/Animations/Walk.fbx is clearly an animation, while Knight/Models/Knight.fbx is a model, even though they both share the same format.The issue with this is that it only works well in one direction. So while you might already know what an asset is for when given its path, you can’t deduce its path if only given information on what the asset does. Being able to find an asset – for example, the material for a character – is useful when trying to automate the setup for some aspects of the assets. While this can be solved by using a rigid naming convention to ensure that the path is easy to deduce, it’s still susceptible to error. Even if you remember the convention, typos are common.An interesting approach to solve this is by using labels. You can use an Editor script that parses the paths of assets and assigns them labels accordingly. As the labels are automated, it’s possible to figure out the exact label that an asset will have. You can even look up assets by their label using AssetDatabase.FindAssets.If you want to automate this sequence, there is a class that can be very handy called the AssetPostprocessor. The AssetPostprocessor receives various messages when Unity imports assets. One of those is OnPostprocessAllAssets, a method that’s called whenever Unity finishes importing assets. It will give you all the paths to the imported assets, providing an opportunity to process those paths. You can write a simple method, like the following, to process them:In the case of the prototype, let’s focus on the list of imported assets – both to try and catch new assets, as well as moved assets. After all, as the path changes, we might want to update the labels.To create the labels, parse the path and look for relevant folders, prefixes, and suffixes of the name, as well as the extensions. Once you have generated the labels, combine them into a single string and set them to the asset.To assign the labels, load the asset using AssetDatabase.LoadAssetAtPath, then assign its labels with AssetDatabase.SetLabels.Remember, it’s important to only set labels if they have actually changed. Setting labels will trigger a reimport of the asset, so you don’t want this to happen unless it’s strictly necessary.If you check this, then the reimport won’t be an issue: Labels are set the first time you import an asset and saved in the .meta file, which means they’re also saved in your version control. A reimport will only be triggered if you rename or move your assets.With the above steps complete, all assets are automatically labeled, as in the example pictured below.Importing textures into a project usually involves tweaking the settings for each texture. Is it a regular texture? A normal map? A sprite? Is it linear or sRGB? If you want to change the settings of an asset importer, you can use the AssetPostprocessor once more.In this case, you’ll want to use the OnPreprocessTexture message, which is called right before importing a texture. This allows you to change the settings of the importer.When it comes to selecting the right settings for every texture, you need to verify what type of textures you’re working with – which is exactly why labels are key in the first step.With this information, you can write a simple TexturePreprocessor:It’s important to ensure that you only run this for textures that have the art label. You’ll then get a reference to the importer so that you can set everything up – starting with the texture size.The AssetPostprocessor has a context property from which you can determine the target platform. As such, you can complete platform-specific changes, like setting the textures to a lower resolution for mobile:Next, check the label to see if the texture is a UI texture, and set it accordingly:For the rest of the textures, set the values to a default. It’s worth noting that Albedo is the only texture that will have sRGB enabled:Thanks to the above script, when you drag and drop the new textures into the Editor, they will automatically have the right settings in place.“Channel packing” refers to the combination of diverse textures into one by using the different channels. It is common and offers many advantages. For instance, the value of the Red channel is metallic and the value of the Green channel is its smoothness.However, combining all textures into one requires some extra work from the art team. If the packing needs to change for some reason, the art team will have to redo all the textures that are used with that shader.As you can see, there’s room for improvement here. The approach that I like to use for channel packing is to create a special asset type where you set the “raw” textures and generate a channel-packed texture to use in your materials.First, I create a dummy file with a specific extension and then use a Scripted Importer that does all the heavy lifting when importing that asset. This is how it works:The importers can have parameters, such as the textures you need to combine.From the importer, you can set the textures as a dependency, which allows the dummy asset to be reimported every time one of the source textures changes. This lets you rebuild the generated textures accordingly.The importer has a version. If you need to change the way that textures are packed, you can modify the importer and bump the version. This will force a regeneration of all the packed textures in your project and everything will be packed in the new way, immediately.A nice side effect of generating things in an importer is that the generated assets only live in the Library folder, so it doesn’t fill up your version control.To implement this, create a ScriptableObject that will hold the created textures and serve as the result of the importer. In the example, I called this class TexturePack.With this created, you can begin by declaring the importer class and adding the ScriptedImporterAttribute to define the version and extension associated with the importer:In the importer, declare the fields you want to use. They will appear in the Inspector, just as MonoBehaviours and ScriptableObjects do:With the parameters ready, create new textures from the ones you have set as parameters. Note, however, that in the Preprocessor, we set isReadable to True to do this.In this prototype, you’ll notice two textures: the Albedo, which has the Albedo in the RGB and a mask for applying the player color in the Alpha, and the Mask texture, which includes the metallic in the Red channel and the smoothness in the Green channel.While this is perhaps outside the scope of this article, let’s look at how to combine the Albedo and the player mask as an example. First, check to see if the textures are set, and if they are, get their color data. Then set the textures as dependencies using AssetImportContext.DependsOnArtifact. As mentioned above, this will force the object to be recalculated if any of the textures end up changing.You also need to create a new texture. To do this, get the size from the TexturePreprocessor that you created in the previous section so that it follows the preset restrictions:Next, fill in all the data for the new texture. This could be massively optimized by using Jobs and Burst. Here we’ll use a simple loop:Set this data in the texture:Now, you can create the method for generating another texture in a very similar way. Once this is ready, create the main body of the importer. In this case, we’ll only create the ScriptableObject that holds the results, creates the textures, and sets the result of the importer through the AssetImportContext.When you write an importer, all of the assets generated must be registered using AssetImportContext.AddObjectToAsset so that they appear in the project window. Select a main asset using AssetImportContext.SetMainObject. This is what it looks like:The only thing left to do is to create the dummy assets. As these are custom, you can’t use the CreateAssetMenuattribute. You must make them manually instead.Using the MenuItem attribute, specify the full path to the create the asset menu, Assets/Create. To create the asset, use ProjectWindowUtil.CreateAssetWithContent, which generates a file with the content you’ve specified and allows the user to input a name for it. It looks like this:Finally, create the channel-packed textures.Most projects use custom shaders. Sometimes they’re used to add extra effects, like a dissolve effect to fade out defeated enemies, and other times, the shaders implement a custom art style, like toon shaders. Whatever the use case, Unity will create new materials with the default shader, and you will need to change it to use the custom shader.In this example, the shader used for units has two added features: the dissolve effect and the player color. When implementing these in your project, you must ensure that all the buildings and units use the appropriate shader.To validate that an asset matches certain requirements – in this case, that it uses the right shader – there is another useful class: the AssetModificationProcessor. With AssetModificationProcessor.OnWillSaveAssets, in particular, you’ll be notified when Unity is about to write an asset to disk. This will give you the opportunity to check if the asset is correct and fix it before it’s saved.Additionally, you can “tell” Unity not to save the asset, which is effective for when the problem you detect cannot be fixed automatically. To accomplish this, create the OnWillSaveAssets method:To process the assets, check whether they are materials and if they have the right labels. If they match the code below, then you have the correct shader:What’s convenient here is that this code is also called when the asset is created, meaning the new material will have the correct shader.As a new feature in Unity 2022, we also have Material Variants. Material Variants are incredibly useful when creating materials for units. In fact, you can create a base material and derive the materials for each unit from there – overriding the relevant fieldsand inheriting the rest of the properties. This allows for solid defaults for our materials, which can be updated as needed.Importing animations is similar to importing textures. There are various settings that need to be established, and some of them can be automated.Unity imports the materials of all the FBXfiles by default. For animations, the materials you want to use will either be in the project or in the FBX of the mesh. The extra materials from the animation FBX appear every time you search for materials in the project, adding quite a bit of noise, so it’s worth disabling them.To set up the rig – that is, choosing between Humanoid and Generic, and in cases where we are using a carefully setup avatar, assigning it – apply the same approach that was applied to textures. But for animations, the message you’ll use is AssetPostprocessor.OnPreprocessModel. This will be called for all FBX files, so you need to discern animation FBX files from model FBX files.Thanks to the labels you set up earlier, this shouldn’t be too complicated. The method starts much like the one for textures:Next up, you’ll want to use the rig from the mesh FBX, so you need to find that asset. To locate the asset, use the labels once more. In the case of this prototype, animations have labels that end with “animation,” whereas meshes have labels that end with “model.” You can complete a simple replacement to get the label for your model. Once you have the label, find your asset using AssetDatabase.FindAssets with “l:label-name.”When accessing other assets, there’s something else to consider: It’s possible that, in the middle of the import process, the avatar has not yet been imported when this method is called. If this occurs, the LoadAssetAtPath will return null and you won’t be able to set the avatar. To work around this issue, set a dependency to the path of the avatar. The animation will be imported again once the avatar is imported, and you will be able to set it there.Putting all of this into code will look something like this:Now you can drag the animations into the right folder, and if your mesh is ready, each one will be set up automatically. But if there isn’t an avatar available when you import the animations, the project won’t be able to pick it up once it’s created. Instead, you’ll need to reimport the animation manually after creating it. This can be done by right-clicking the folder with the animations and selecting Reimport.You can see all of this in the sample video below.Using exactly the same ideas from the previous sections, you’ll want to set up the models you are going to use. In this case, employ AssetPostrocessor.OnPreprocessModel to set the importer settings for this model.For the prototype, I’ve set the importer to not generate materialsand checked whether the model is a unit or a building. The units are set to generate an avatar, but the avatar creation for the buildings is disabled, as the buildings aren’t animated.For your project, you might want to set the materials and animatorswhen importing the model. This way, the Prefab generated by the importer is ready for immediate use.To do this, use the AssetPostprocessor.OnPostprocessModel method. This method is called after a model is finished importing. It receives the Prefab that has been generated as a parameter, which lets us modify the Prefab however we want.For the prototype, I found the material and Animation Controller by matching the label, just as I located the avatar for the animations. With the Renderer and Animator in the Prefab, I set the material and the controller as in normal gameplay.You can then drop the model into your project and it will be ready to drop into any scene. Except we haven’t set any gameplay-related components, which I’ll address in the second part of this blog.With these advanced scripting tips, you’re just about game ready. Stay tuned for the next installment in this two-part Tech from the Trenches article, which will cover hacks for balancing game data and more.If you would like to discuss the article, or share your ideas after reading it, head on over to our Scripting forum. You can also connect with me on Twitter at @CaballolD. #advanced #editor #scripting #hacks #save

Hello I’m trying to get substrate refraction to work, so far it’s going well and it’s working the only issue I’m having I can’t get zero refraction. If you have a black and white mask and you want refraction only in the white parts how can I zero refraction wherever there’s black in the mask? For me it’s leaving behind a very subtle dark tint What I’ve done is, I have a Substrate Slab BSDF - Simple. In Roughness I have 0, in SSS MFP I have 1, in Normal I have a Normal map, in Refraction I have a Lerp A is 1, B is 0 and Alpha is a mask texture. However there’s a subtle very subtle black tint where the black of the mask is in, I don’t know how to completely remove that dark tint, it’s very subtle but it’s visible nonetheless. #substrate #material #refraction #help

NPR Project May 23rd, 2025 Code Design, General Development Clément Foucault html PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN" "; Wing it! Early NPR project by Blender Studio. In July 2024 the NPRproject officially started, with a workshop with Dillo Goo Studio and Blender developers. While the use-cases were clear, the architecture and overall design were not. To help with this, the team started working in a prototype containing many shading features essential to the NPR workflow. This prototype received a lot of attention, with users contributing a lot of nice examples of what is possible with such system. The feedback showed that there is a big interest from the community for a wide range of effects. However the amount of flexibity made possible with the prototype came with a cost: it locked NPR features within EEVEE, alienating Cycles from part of the NPR pipeline. It also deviated from the EEVEE architecture, which could limit future feature development. After much consideration, the design was modified to address these core issues. The outcome can be summarized as: Move filters and color modification to a multi-stage compositing workflow. Keep shading features inside the renderer’s material system. Multi-stage compositing One of the core feature needed for NPR is the ability to access and modify the shaded pixels. Doing it inside a render engine has been notoriously difficult. The current way of doing it inside EEVEE is to use the ShaderToRGB node, which comes with a lot of limitations. In Cycles, limited effects can be achieved using custom OSL nodes. As a result, in production pipeline, this is often done through very cumbersome and time consuming scene-wide compositing. The major downside is that all asset specific compositing needs to be manually merged and managed inside the scene compositor. Instead, the parts of the compositing pipeline that are specific to a certain asset should be defined at the asset level. The reasoning is that these compositing nodes define the appearance of this asset and should be shared between scene. Multi-stage compositing is just that! A part of the compositing pipeline is linked to a specific object or material. This part receives the rendered color as well as its AOVs and render passes as input, and output the modified rendered color. The object level compositor at the bottom right define the final appearance of the object In this example the appearance of the Suzanne object is defined at the object level inside its asset file. When linked into a scene with other elements, it is automatically combined with other assets. From left to right: Smooth Toon shading with alpha over specular, Pixelate, Half-Tone with Outline This new multi-stage compositing will be reusing the compositor nodes, with a different subset of nodes available at the object and material levels. This is an opportunity to streamline the workflow between material nodes editing and compositor nodes. Grease Pencil Effects can eventually be replaced by this solution. Final render showing 3 objects with different stylizations seamlessly integrated. There are a lot more to be said about this feature. For more details see the associated development task. Anti-Aliased output A major issue with working with the a compositing workflow is Anti-Aliasing. When compositing anti-aliased input, results often include hard to resolve fringes. Left: Render Pass, Middle: Object Matte, Right: Extracted Object Render Pass The common workaround to this issue is to render at higher resolution without AA and downscale after compositing. This method is very memory intensive and only allows for 4x or 9x AA with usually less than ideal filtering. Another option is to use post-process AA filters but that often results in flickering animations. Left: Anti-Aliased done before compositor based shadingRight: Anti-Aliasing is done after compositor. The solution to this problem is to run the compositor for each AA step and filter the composited pixels like a renderer would do. This will produce the best image quality with only the added memory usage of the final frame. Converged input One of the main issues with modern renderers is that their output is noisy. This doesn’t play well with NPR workflows as many effects require applying sharp transformations of the rendered image or light buffers. For instance, this is what happens when applying a constant interpolated color ramp over the ambient occlusion node. The averaging operation is run on a noisy output instead of running on a noisy input before the transformation. Left: Original AO, Middle: Constant Ramp in material, Right: Ramp applied in compositorDoing these effects at compositing time gives us the final converged image as input. However, as explained above, the compositor needs to run before the AA filtering. So the multi-stage compositors needs to be able to run on converged or denoised inputs while being AA free. In other words, it means that the render samples will be distributed between render passes convergence and final compositor AA. Engine Features While improving the compositing workflow is important for stylization flexibility, some features are more suited for the inside of the render engine. This allows builtin interaction with light transport and other renderer features. These features are not exclusive to NPR workflows and fit well inside the engine architecture. As such, the following features are planned to be directly implemented inside the render engines: Ray Queries Portal BSDF Custom Shading Depth Offset The development will start after the Blender 5.0 release, planned for November 2025. Meanwhile, to follow the project, subscribe to the development task. For more details about the project, join the announcement thread. Support the Future of Blender Donate to Blender by joining the Development Fund to support the Blender Foundation’s work on core development, maintenance, and new releases. ♥ Donate to Blender #npr #project

Game Developer tinybugbot, whose Houdini stuff we frequently feature on 80 Level, shared an example of how to use a mask with an SDF stored in a VDB to deform geometry. While they prefer Wrangle nodes, you can also experiment with OpenCL, which operates at a lower level than VEX and can offer better performance depending on your hardware and use case.tinybugbot has also previously shared tips on creating displacement effects and designing mechanical surface textures:Also, check out procedural cliffs, a fried egg, and more great projects on their X/Twitter:Join our 80 Level Talent platform and our new Discord server, follow us on Instagram, Twitter, LinkedIn, Telegram, TikTok, and Threads, where we share breakdowns, the latest news, awesome artworks, and more. #deforming #geometry #with #donutshaped #sdf

RT Karl PoyzerMore SDF fun with this alien gadget thing. Inspired by the work of @noaxdesign #b3d #karl #poyzer #more #sdf #fun

Interviews Daredevil – Born Again: Gong Myung Lee – Production VFX Supervisor By Vincent Frei - 15/05/2025 With over two decades of experience in visual effects, Gong Myung Lee started her career in 2003 and went on to work at renowned studios like Mr. X and Method Studios. She became part of the Marvel Studios team in 2022. Her credits include The Finest Hours, Deadpool 2, Triple Frontier, and The Changeling. What is your background? I started in VFX as a Lighting Technical Director and Compositor, which gave me a deep appreciation for how images work—light, depth, and how all the parts come together. From there, I moved into CG supervision and eventually into VFX supervision, working across features, commercials, and episodic projects. I’ve worked both on the vendor side and in production. At facilities, I built and led teams and tackled creative challenges from inside the post pipeline. Shifting to the production side let me be involved earlier—helping shape decisions on set and ensuring VFX supports the storytelling from day one. How did you get involved on this show? I’d always hoped to work with Marvel on the production side, so when Jen Underdahl called me about this opportunity, I said yes immediately. I’ve been a fan of Daredevil for years—his internal conflict, his vulnerability, and the constant pull between justice and vengeance. And the fact that it was shooting entirely in New York made it even more exciting. Daredevil: Born Again has a distinct visual style. How did you collaborate with the showrunner and directors to ensure that the VFX seamlessly fit within their creative vision? From the start, it was clear this wasn’t going to be a typical superhero show. No lasers, no flying, no giant CG creatures. Daredevil is a street-level hero—he gets hurt, he bleeds, and he wrestles with morality and faith. The VFX had to reflect that same grounded humanity. Everything needed to feel physical, real, and restrained. We worked closely with showrunner Dario Scardapane, directors Justin Benson and Aaron Moorhead, and Marvel creative executive Sana Amanat to define a visual language built around subtlety. The VFX weren’t there to impress—they were there to disappear into the world. We also pulled from key comic book imagery that Sana wanted to preserve—Daredevil crouched on a rooftop, his mid-air baton throws, the skyline behind him. Those became touchpoints for our design language. Being involved from preproduction through delivery meant we could plan closely with every department—art, stunts, SFX, camera—so that VFX could step in where needed. We captured as much in camera as possible, using VFX for what couldn’t be achieved safely or practically. That ranged from invisible stitches to CG weapons, digi-doubles, and full digital builds. Because we had multiple directors and DPs, VFX became a continuity backbone—carrying consistent design and tone across episodes. That applied to everything from the sensory visuals and BB Report graphics, to how Daredevil’s baton behaved from one fight to the next. We also tracked the emotional and visual arc of the show. The intensity of violence escalates as the story deepens. In Episode 102, when Daredevil breaks a cop’s arm, we held back—the blood simply soaks through the sweater, staying grounded in realism. By Episode 109, when Fisk crushes Gallo’s head, the moment is fully on camera. That evolution was intentional. To make it all sit inside the frame, we were rigorous with data acquisition. Every set was LiDAR scanned with photogrammetry, and effects-heavy scenes were supported with HDRIs, cyberscans, clean plates, and lens grids. The goal wasn’t just coverage—it was making sure our work belonged inside the photography. Were there specific requests from the directors regarding the way VFX should enhance the realism and grittiness of the series, particularly in terms of lighting, camera movement, or fight choreography? Absolutely. From the get-go, directors Justin Benson and Aaron Moorhead established a clear visual rulebook. Camera language was closely tied to character: Fisk’s scenes leaned into locked-off frames and symmetry, while Murdock’s were more handheld and reactive. Zooms were preferred over push-ins, and dutch angles were generally avoided. Even when VFX took over a shot—like a CG camera move—we stuck to those same principles to preserve the tone. Our DP, Hillary Spera, shot on Alexa 35s using custom-tweaked anamorphic lenses, which created beautiful lens flares and optical distortions. We mapped those lenses and captured a full library of flare elements so that VFX could match them precisely. That look carried through to DI and helped define the show’s distinct texture. Atmosphere—especially fog and wetdowns—was critical to the show’s look. It shaped lighting and grounded the environments. But in shots requiring CG or 2D elements, we often recreated matching FX atmosphere in post to ensure seamless interaction and integration into the plate. Action followed the same grounded approach. Our Action Director and Supervising Stunt Coordinator, Philip Silvera, choreographed fights based on real-world combat logic, but with emotional weight built into the pacing—breath, exhaustion, impact. For digital takeovers, we started with mocap of stunt performances, then refined using video reference to match the actors’ specific movement and intention. We didn’t shy away from gore either. The directive was clear: keep it grounded, but don’t pull punches. We referenced surgical and forensic material to guide the look of blood hits, wounds, and trauma. Practical prosthetics and squibs formed the base, with VFX adding layers—muscle tear, splatter, and dynamic blood flow—to heighten realism without drifting into stylization. Photo credit: Giovanni Rufino How did you choose the various vendors and split the work amongst them? VFX is a team sport—and like any good team, you need to know your players and play to their strengths. Every facility brings something unique, and a big part of our early conversations was about assigning work based on where each vendor excels. It was about setting everyone up for success. The best VFX comes from the artist’s eye and hand. That final 5%—the nuance and polish—is where the work really comes alive. So I looked for teams who not only had the technical chops but understood the tone of the show and cared deeply about the craft. When that alignment happens, the collaboration just works. We planned the vendor split early, ensuring shared assets would work across facilities and that pipelines were compatible. The Third Floor handled previs, techviz, and postviz. LolaVFX supported early look development. RISE took on some of the most technically complex sequences—like the opening oner at Josie’s, digi-double transitions, and full environment builds. Folks VFX delivered rooftop environments, the finale apartment fight, and key continuity-heavy scenes. Ghost VFX handled the subway fight and Daredevil’s tunnel run, while Phosphene developed the sensory language and crafted bespoke moments like the blackout sequence and car comps. Soho VFX and Powerhouse VFX focused on fight choreography, weapon augmentation, and full-CG sequences. Cantina Creative created all in-world monitor graphics and interfaces. Anibrain, SDFX, and Base FX tackled difficult cleanups, split comps, and continuity patches. Dark Red Studios handled the BB Report design. Scanable handled all LiDAR and cyberscans across our sets and cast—crucial for digi-double and environment work. And we also had an internal team dedicated to the more subtle sensory shots. It was a true team effort, and the final result reflects the strength and dedication each vendor brought to the table. Photo credit: Giovanni Rufino The opening sequence is an incredibly complex oner action scene. Can you walk us through the planning and VFX techniques used to stitch everything together seamlessly? This sequence was a massive effort across all departments, and for VFX, it was one of the most complex and rewarding challenges of the season. The scene begins with Daredevil and Bullseye crashing through the window of Josie’s Bar, moves into the back room, up through a stairway, and eventually emerges onto a rooftop. None of those locations existed together physically, so stitching them into one continuous action required close collaboration between stunts, camera, SFX, and VFX from day one. Our stunt team, led by Philip Silvera, provided a detailed stuntviz that broke the choreography into stitchable beats, which became our foundation. Directors Justin Benson and Aaron Moorhead wanted the camera to feel intentional and fluid—no fast whips or chaotic edits—so every transition had to be precise. There was nowhere to hide a bad seam. RISE led the VFX for the sequence, using a blend of techniques—2D wipes with foreground elements, bluescreen set extensions, FX smoke, and CG transitions—but the real connective tissue was in 3D. We LiDAR-scanned all key locations and rebuilt them in CG, allowing us to bridge multiple stages, extend environments, and support digi-double and FX work with accuracy. Stitching those spaces and action beats required careful spatial continuity and character animation. We used motion capture as the base for digital double work, then enhanced it with keyframed animation to preserve choreography and hit those iconic silhouettes that are signature to Daredevil and Bullseye. The camera tilt moment—where we drop down to see Foggy on the street and then tilt back up to Daredevil and Bullseye—was a digital blend using a 2.5D street projection and a full-CG building facade. That allowed us to link two separate physical locations with one fluid camera move. SFX, supervised by our SFX Coordinator Roy Savoy, handled all the glass breaks and practical sparks inside Josie’s, while Props took care of the breakaway items. Together, they gave us strong in-camera interaction points that VFX could build on. We layered in additional CG debris, neon signage, and more sparks to push the action further. The smoke and atmosphere—inside the bar and on the rooftop—were fully CG to allow precise interaction with digital characters, weapons, gore, and FX. Everything—smoke, blood hits, atmosphere, digi-double transitions—had to feel cohesive and continuous. The real achievement was making the stitching and technical complexity disappear. Given the mix of practical stunts and digital enhancements, what were the biggest challenges in maintaining continuity and realism throughout this oner? One of the biggest challenges was maintaining character integrity—especially for Daredevil. Charlie brings such a specific physicality to the role that even under smoke or in silhouette, his movement reads clearly. Every digital double had to mirror that language—his timing, stance, and how he moves through space—so the audience never questioned the switch. RISE used stunt mocap as the foundation for the digital doubles, then layered in character-specific adjustments to preserve that continuity. At the same time, we were building out full-CG environments to bridge shots, which meant matching textures, lighting, geometry, and camera lensing so that transitions were seamless. Weapons were another layer. Daredevil’s baton, Bullseye’s firearms and knives—all were CG. In one sequence, Daredevil is stabbed by 17 blades, some of which he pulls out to fight back. We had to track each knife’s position, contact shadow, suit tear, and blood drip from shot to shot, which required full-body rotomation and progressive damage built into the suit asset. That also extended to the environment—any damage had to be tracked through the sequence. Grounding the action was essential. Even when Daredevil lands from a rooftop with his grappling hook, it had to feel weighty. We added dust bursts, imperfect footfalls, and a bit of camera vibration to keep it visceral. In a sequence this seamless, every movement, every transition, every detail had to hold up. The illusion depended on total cohesion. How did you approach the digital recreation of New York and Hell’s Kitchen to match the series’ darker, more grounded aesthetic? New York is more than a backdrop—it’s part of Daredevil’s identity. So our approach to the digital builds had to feel authentic, responsive, and emotionally aligned with the show’s tone. We LiDAR-scanned full city blocks, rooftops, and adjacent buildings, and captured tiled plates and HDRIs throughout the day to map real lighting conditions. This gave us a strong base to work from, while still allowing flexibility for framing and layout. Folks VFX handled Matt’s rooftop in Episode 103, while RISE built the Josie’s rooftop and surrounding Hell’s Kitchen. Both spaces had to feel true to the city—textured and believable. Even slight changes in building height or placement could shift the tone of the scene, so we were careful to maintain a balance between realism and composition. Atmosphere was a defining element. We added fog, steam, wetdowns, reflections, and other city-layered effects—like flickering windows and distant traffic—to give each environment weight and depth. These elements weren’t just aesthetic—they helped shape the lighting and made our digital spaces feel physically inhabited and consistent with the show’s tone. Were there any key locations that had to be fully CG or heavily augmented with VFX? If so, what was the process of integrating them into the live-action footage? Yes—several locations had to be fully CG or significantly augmented. Matt’s apartment rooftop in Williamsburg, the Hell’s Kitchen rooftop at Josie’s, the fire escape sequence, Hector’s subway fight, Daredevil’s tunnel run, and the sensory bank vault scene all required extensive digital builds. We began by scanning or photographing real-world references, scouted in collaboration with our creative team, Production Designer and DP. Using LiDAR and photogrammetry, we rebuilt those environments in 3D, preserving scale and lighting using on-set HDRIs. Foreground and midground buildings were modeled in high detail, and wherever windows were visible, we added interior structure and depth to maintain a sense of life beyond the facade. In some cases, city regulations or safety concerns limited our ability to fully scan practical locations. For sequences like the subway platform and the tunnel Daredevil runs through in Episode 106, we used partial scans and reference photography to reconstruct and extend those environments in CG—including the train asset and the tunnel’s structural continuity. In Episode 105, the bank vault sensory sequence was shot on stage with only the real vault door present. Everything else—the spatial layout, dimensionality, and lighting—was designed in CG to reflect the show’s established visual language for Daredevil’s heightened perception. Not exactly an environment extension, but for the “blackout” sequences in the finale, we had to heavily modify our plates to feel like a true city-wide outage. That involved blending day-for-night captures with digital relighting in comp. In Times Square, we replaced the practical LED content with FISK delivering his New Year’s Eve speech. And to build tension, we expanded Fisk’s motorcade in Red Hook using CG mocaped taskforce units and Humvees. Whether it was a full CG build or a heavy plate modification, every element was grounded back into the photography through precise matchmove, consistent lighting cues, and environmental reference. Marvel projects often have a mix of on-location and green-screen work. What techniques did you use to ensure a seamless blend between real and digital environments? The key was always grounding the visual effects in something tangible. Whether we were working on a full bluescreen stage or extending a rooftop, the more real-world reference we could gather, the more seamlessly the VFX would sit in the frame. On Daredevil: Born Again, we captured practical plates and tiled reference footage wherever possible—especially throughout New York. That gave us a photographic base to match to. From there, we built every VFX setup on solid acquisition: LiDAR scans, high-res photogrammetry, HDRIs for lighting, and texture capture from all angles. We avoided relying entirely on CG whenever we could. Even in the most complex shots, we tried to include a practical element—whether it was a foreground prop, atmospheric haze, wet-downs or interactive lighting on set. Those physical components gave us something real to build off and helped the audience feel rooted in the scene. For environments, 2.5D extensions were backed by real plates, and full-CG builds were constructed from real-world geometry and surface data. Even on bluescreen stages, we lit to match the look and feel of the corresponding practical scenes to preserve continuity. We also worked closely with DI to carry through the grade and environmental treatment. That final layer ensured the shot didn’t just blend technically—but emotionally felt like it belonged to the same world. Daredevil is known for its brutal, grounded action sequences. How did VFX help enhance the stunt work while keeping it as practical as possible? Our approach was always practical-first. If it could be done in camera—blood hits, breakaways, physical impacts—we shot it that way. Then VFX would step in to enhance what was already there and bring the moment to full fidelity. For example, blood on faces and inside mouths was done practically using SFX makeup. But for continuity or when choreography called for additional impact, we added digital blood sprays, squibs, or drips—carefully tracked to the performance so they felt embedded, not layered on. Our stunt team gave us strong reference across all key fight beats. We combined that with mocap when needed, especially in sequences that transitioned into digital doubles. Having that real-world base allowed us to maintain physicality and character intention throughout. We also captured a library of flares and lens artifacts from our camera package, giving VFX the right visual language to match the look and feel of the photography—especially during close-quarters combat, where blood, smoke, and light moved dynamically through frame. It helped reinforce the tactile, in-camera aesthetic, even in fully augmented shots. Were there any particular fight sequences that required extensive digital double work, and how did you ensure they remained indistinguishable from the real actors? There were definitely moments where digi-doubles became essential—sometimes for safety, sometimes to maintain camera continuity, and sometimes because the physical environment couldn’t support the choreography as scripted. Sequences like Daredevil’s rooftop runs, tunnel chases, fire escape landings, grappling hook jumps, transitions within the oner, and background extensions to enhance Fisk’s task force all required digital double work at varying levels of complexity. In every action scene, our approach was to ground the work in real-world performance. We captured motion for all key stunts and fight beats, and did detailed cyberscans of the main cast and their stunt doubles. That gave us accurate silhouettes, proportions, and suit detail—everything we needed to maintain visual continuity between live-action and CG. While we didn’t use facial capture rigs or volume setups, Charlie Cox gave us highly expressive reference performances. His focused takes gave us insight into the intention behind each movement, which we either projected in 2.5D or matched in animation for the digi-double moments. What really sold it was the integration. We were meticulous about matching HDRI lighting, lensing, atmosphere, and even subtle depth-of-field cues. Every digital element had to feel like it was captured on the same lens, in the same space, at the same time. That’s what kept the illusion seamless—even in full-CG moments. Were there any memorable moments or scenes from the series that you found particularly rewarding or challenging to work on from a visual effects standpoint? One that stands out is the rooftop environment for Matt’s apartment. It was our first major VFX presentation to the directors and creatives, and it set the tone for what our digital New York would become. It needed to feel cinematic, atmospheric, and emotionally grounded. We scanned and photographed rooftops around Williamsburg, then composited those elements into a custom layout tailored for our scenes. But even subtle changes—like shifting a building’s height or shape—could disrupt the familiarity of the skyline. It’s like adjusting features on a face: subtle, but if it’s wrong, you feel it. Getting that balance right, while still supporting our camera moves and blocking, was incredibly rewarding. Another favorite was developing Daredevil’s sensory language, which we crafted in close collaboration with Phosphene. It evolved throughout the season, but the most ambitious moment was what we called the “Grande Sensory” shot—when Daredevil tunes into a specific sound and his world expands perceptually. We shot it using a custom three-camera rig: a spherical zoom lens in the center, flanked by wide primes. As we pushed in while zooming out—a zolly—we stitched the feeds together at the widest focal length into an approximately 230° field of view, with extra image gathering on the sides. Simultaneously, the aspect ratio expands. Once Daredevil locks onto the target sound, we “unstitch” back to the center camera and compress the aspect ratio as we zoom in for a clean finish. The transitions are subtle but emotionally impactful—designed to be felt, not noticed. We also had the “petite sensory” moments—simpler, analog-feeling visual cues that suggest Daredevil is focusing in. These used optical lens tricks, subtle distortions, and flares—many based on practical elements we shot—so even the heightened perception moments felt grounded in the show’s visual language. And finally, the Gallo head-crush—one of the season’s most graphic shots. The practical prosthetic was impressive, but once pressure was applied, it deformed in a way that didn’t mimic human anatomy. We scanned both the actor and prosthetic for a partial digital takeover—replacing areas with CG muscle, bone, blood, and facial details. Folks VFX handled this sequence and absolutely nailed it, especially the vascular bursts in the eyes and the blink just before the fade. Looking back on the project, what aspects of the visual effects are you most proud of? There’s a lot to be proud of, but what stands out most to me is the consistency of tone and craft—how the visual effects supported the grounded, emotional feel of the show without ever pulling focus. Even in sequences that were technically complex—like the rooftop builds, sensory moments, or the opening oner—we always came back to story. The VFX weren’t about spectacle. They were about precision, intention, and integration. I’m also proud of the continuity we maintained across multiple directors and DPs. Visual effects became a steady thread—carrying design language, violence levels, atmosphere, and choreography from one episode to the next. Most of all, I’m proud that the work feels cohesive. Whether it’s a subtle stitch or a full CG shot, it never distracts. It feels lived in—part of the world, not outside of it. How long have you worked on this show? I was on from November 2022 through November 2024, so a full two years. Being involved early really helped—we had time to plan things properly, make smart choices, and stay ahead instead of playing catch-up. What’s the VFX shot count? We delivered 1,380 final shots in the official turnover cut. What is your next project? We’re already shooting Season 2 of Daredevil: Born Again, which has been exciting. Coming back with the same core team means there’s already a rhythm in place—we know how to work together, and that gives us room to push things further. We’re building on what we set up in Season 1, keeping the tone grounded, but also exploring new visual ideas and storytelling challenges. It’s great to be back in this world, with a chance to evolve it in new ways. What are the four movies that gave you the passion for cinema? There are many, but ones that come to mind immediately are The Godfather I & II, The Shining, In the Mood for Love, and Dune: Part One & Part Two. A big thanks for your time. WANT TO KNOW MORE?RISE: Dedicated page about Daredevil: Born Again on RISE website. © Vincent Frei – The Art of VFX – 2025 #daredevil #born #again #gong #myung

Bagging and boosting are two powerful ensemble techniques in machine learning – they are must-knows for data scientists! After reading this article, you are going to have a solid understanding of how bagging and boosting work and when to use them. We’ll cover the following topics, relying heavily on examples to give hands-on illustration of the key concepts: How Ensembling helps create powerful models Bagging: Adding stability to ML models Boosting: Reducing bias in weak learners Bagging vs. Boosting – when to use each and why Creating powerful models with ensembling In Machine Learning, ensembling is a broad term that refers to any technique that creates predictions by combining the predictions from multiple models. If there is more than one model involved in making a prediction, the technique is using ensembling! Ensembling approaches can often improve the performance of a single model. Ensembling can help reduce: Variance by averaging multiple models Bias by iteratively improving on errors Overfitting because using multiple models can increase robustness to spurious relationships Bagging and boosting are both ensemble methods that can perform much better than their single-model counterparts. Let’s get into the details of these now! Bagging: Adding stability to ML models Bagging is a specific ensembling technique that is used to reduce the variance of a predictive model. Here, I’m talking about variance in the machine learning sense – i.e., how much a model varies with changes to the training dataset – not variance in the statistical sense which measures the spread of a distribution. Because bagging helps reduce an ML model’s variance, it will often improve models that are high variancebut won’t do much good for models that are low variance. Now that we understand when bagging helps, let’s get into the details of the inner workings to understand how it helps! The bagging algorithm is iterative in nature – it builds multiple models by repeating the following three steps: Bootstrap a dataset from the original training data Train a model on the bootstrapped dataset the trained model The collection of models created in this process is called an ensemble. When it is time to make a prediction, each model in the ensemble makes its own prediction – the final bagged prediction is the averageor majority voteof all of the ensemble’s predictions. Now that we understand how bagging works, let’s take a few minutes to build an intuition for why it works. We’ll borrow a familiar idea from traditional statistics: sampling to estimate a population mean. In statistics, each sample drawn from a distribution is a random variable. Small sample sizes tend to have high variance and may provide poor estimates of the true mean. But as we collect more samples, the average of those samples becomes a much better approximation of the population mean. Similarly, we can think of each of our individual decision trees as a random variable — after all, each tree is trained on a different random sample of the data! By averaging predictions from many trees, bagging reduces variance and produces an ensemble model that better captures the true relationships in the data. Bagging Example We will be using the load_diabetes1 dataset from the scikit-learn Python package to illustrate a simple bagging example. The dataset has 10 input variables – Age, Sex, BMI, Blood Pressure and 6 blood serum levels. And a single output variable that is a measurement of disease progression. The code below pulls in our data and does some very simple cleaning. With our dataset established, let’s start modeling! # pull in and format data from sklearn.datasets import load_diabetes diabetes = load_diabetesdf = pd.DataFramedf.loc= diabetes.target df = df.dropnaFor our example, we will use basic decision trees as our base models for bagging. Let’s first verify that our decision trees are indeed high variance. We will do this by training three decision trees on different bootstrapped datasets and observing the variance of the predictions for a test dataset. The graph below shows the predictions of three different decision trees on the same test dataset. Each dotted vertical line is an individual observation from the test dataset. The three dots on each line are the predictions from the three different decision trees. Variance of decision trees on test data points – image by author In the chart above, we see that individual trees can give very different predictionswhen trained on bootstrapped datasets. This is the variance we have been talking about! Now that we see that our trees aren’t very robust to training samples – let’s average the predictions to see how bagging can help! The chart below shows the average of the three trees. The diagonal line represents perfect predictions. As you can see, with bagging, our points are tighter and more centered around the diagonal. image by author We’ve already seen significant improvement in our model with the average of just three trees. Let’s beef up our bagging algorithm with more trees! Here is the code to bag as many trees as we want: def train_bagging_trees: ''' Creates a decision tree bagging model by training multiple decision trees on bootstrapped data. inputs df: training data with both target and input columns target_col: name of target column pred_cols: list of predictor column names n_trees: number of trees to be trained in the ensemble output: train_trees: list of trained trees ''' train_trees =for i in range: # bootstrap training data temp_boot = bootstrap#train tree temp_tree = plain_vanilla_tree# save trained tree in list train_trees.appendreturn train_trees def bagging_trees_pred: ''' Takes a list of bagged trees and creates predictions by averaging the predictions of each individual tree. inputs df: training data with both target and input columns train_trees: ensemble model - which is a list of trained decision trees target_col: name of target column pred_cols: list of predictor column names output: avg_preds: list of predictions from the ensembled trees ''' x = dfy = dfpreds =# make predictions on data with each decision tree for tree in train_trees: temp_pred = tree.predictpreds.append# get average of the trees' predictions sum_preds =avg_preds =return avg_preds The functions above are very simple, the first trains the bagging ensemble model, the second takes the ensembleand makes predictions given a dataset. With our code established, let’s run multiple ensemble models and see how our out-of-bag predictions change as we increase the number of trees. Out-of-bag predictions vs. actuals colored by number of bagged trees – image by author Admittedly, this chart looks a little crazy. Don’t get too bogged down with all of the individual data points, the lines dashed tell the main story! Here we have 1 basic decision tree model and 3 bagged decision tree models – with 3, 50 and 150 trees. The color-coded dotted lines mark the upper and lower ranges for each model’s residuals. There are two main takeaways here:as we add more trees, the range of the residuals shrinks andthere is diminishing returns to adding more trees – when we go from 1 to 3 trees, we see the range shrink a lot, when we go from 50 to 150 trees, the range tightens just a little. Now that we’ve successfully gone through a full bagging example, we are about ready to move onto boosting! Let’s do a quick overview of what we covered in this section: Bagging reduces variance of ML models by averaging the predictions of multiple individual models Bagging is most helpful with high-variance models The more models we bag, the lower the variance of the ensemble – but there are diminishing returns to the variance reduction benefit Okay, let’s move on to boosting! Boosting: Reducing bias in weak learners With bagging, we create multiple independent models – the independence of the models helps average out the noise of individual models. Boosting is also an ensembling technique; similar to bagging, we will be training multiple models…. But very different from bagging, the models we train will be dependent. Boosting is a modeling technique that trains an initial model and then sequentially trains additional models to improve the predictions of prior models. The primary target of boosting is to reduce bias – though it can also help reduce variance. We’ve established that boosting iteratively improves predictions – let’s go deeper into how. Boosting algorithms can iteratively improve model predictions in two ways: Directly predicting the residuals of the last model and adding them to the prior predictions – think of it as residual corrections Adding more weight to the observations that the prior model predicted poorly Because boosting’s main goal is to reduce bias, it works well with base models that typically have more bias. For our examples, we are going to use shallow decision trees as our base model – we will only cover the residual prediction approach in this article for brevity. Let’s jump into the boosting example! Predicting prior residuals The residuals prediction approach starts off with an initial modeland we calculate the residuals of that initial prediction. The second model in the ensemble predicts the residuals of the first model. With our residual predictions in-hand, we add the residual predictions to our initial predictionand recalculate the updated residuals…. we continue this process until we have created the number of base models we specified. This process is pretty simple, but is a little hard to explain with just words – the flowchart below shows a simple, 4-model boosting algorithm. Flowchart of simple, 4 model boosting algorithm – image by author When boosting, we need to set three main parameters:the number of trees,the tree depth andthe learning rate. I’ll spend a little time discussing these inputs now. Number of Trees For boosting, the number of trees means the same thing as in bagging – i.e., the total number of trees that will be trained for the ensemble. But, unlike boosting, we should not err on the side of more trees! The chart below shows the test RMSE against the number of trees for the diabetes dataset. Unlike with bagging, too many trees in boosting leads to overfitting! – image by author This shows that the test RMSE drops quickly with the number of trees up until about 200 trees, then it starts to creep back up. It looks like a classic ‘overfitting’ chart – we reach a point where more trees becomes worse for the model. This is a key difference between bagging and boosting – with bagging, more trees eventually stop helping, with boosting more trees eventually start hurting! With bagging, more trees eventually stops helping, with boosting more trees eventually starts hurting! We now know that too many trees are bad, and too few trees are bad as well. We will use hyperparameter tuning to select the number of trees. Note – hyperparameter tuning is a huge subject and way outside of the scope of this article. I’ll demonstrate a simple grid search with a train and test dataset for our example a little later. Tree Depth This is the maximum depth for each tree in the ensemble. With bagging, trees are often allowed to go as deep they want because we are looking for low bias, high variance models. With boosting however, we use sequential models to address the bias in the base learners – so we aren’t as concerned about generating low-bias trees. How do we decide how the maximum depth? The same technique that we’ll use with the number of trees, hyperparameter tuning. Learning Rate The number of trees and the tree depth are familiar parameters from bagging– but this ‘learning rate’ character is a new face! Let’s take a moment to get familiar. The learning rate is a number between 0 and 1 that is multiplied by the current model’s residual predictions before it is added to the overall predictions. Here’s a simple example of the prediction calculations with a learning rate of 0.5. Once we understand the mechanics of how the learning rate works, we will discuss the why the learning rate is important. The learning rate discounts the residual prediction before updating the actual target prediction – image by author So, why would we want to ‘discount’ our residual predictions, wouldn’t that make our predictions worse? Well, yes and no. For a single iteration, it will likely make our predictions worse – but, we are doing multiple iterations. For multiple iterations, the learning rate keeps the model from overreacting to a single tree’s predictions. It will probably make our current predictions worse, but don’t worry, we will go through this process multiple times! Ultimately, the learning rate helps mitigate overfitting in our boosting model by lowering the influence of any single tree in the ensemble. You can think of it as slowly turning the steering wheel to correct your driving rather than jerking it. In practice, the number of trees and the learning rate have an opposite relationship, i.e., as the learning rate goes down, the number of trees goes up. This is intuitive, because if we only allow a small amount of each tree’s residual prediction to be added to the overall prediction, we are going to need a lot more trees before our overall prediction will start looking good. Ultimately, the learning rate helps mitigate overfitting in our boosting model by lowering the influence of any single tree in the ensemble. Alright, now that we’ve covered the main inputs in boosting, let’s get into the Python coding! We need a couple of functions to create our boosting algorithm: Base decision tree function – a simple function to create and train a single decision tree. We will use the same function from the last section called ‘plain_vanilla_tree.’ Boosting training function – this function sequentially trains and updates residuals for as many decision trees as the user specifies. In our code, this function is called ‘boost_resid_correction.’ Boosting prediction function – this function takes a series of boosted models and makes final ensemble predictions. We call this function ‘boost_resid_correction_pred.’ Here are the functions written in Python: # same base tree function as in prior section def plain_vanilla_tree: X_train = df_trainy_train = df_traintree = DecisionTreeRegressorif weights: tree.fitelse: tree.fitreturn tree # residual predictions def boost_resid_correction: ''' Creates boosted decision tree ensemble model. Inputs: df_train: contains training data target_col: name of target column pred_col: target column names num_models: number of models to use in boosting learning_rate: discount given to residual predictions takes values between: max depth of each tree model Outputs: boosting_model: contains everything needed to use model to make predictions - includes list of all trees in the ensemble ''' # create initial predictions model1 = plain_vanilla_treeinitial_preds = model1.predictdf_train= df_train- initial_preds # create multiple models, each predicting the updated residuals models =for i in range: temp_model = plain_vanilla_treemodels.appendtemp_pred_resids = temp_model.predictdf_train= df_train-boosting_model = {'initial_model' : model1, 'models' : models, 'learning_rate' : learning_rate, 'pred_cols' : pred_cols} return boosting_model # This function takes the residual boosted model and scores data def boost_resid_correction_predict: ''' Creates predictions on a dataset given a boosted model. Inputs: df: data to make predictions boosting_models: dictionary containing all pertinent boosted model data chart: indicates if performance chart should be created Outputs: pred: predictions from boosted model rmse: RMSE of predictions ''' # get initial predictions initial_model = boosting_modelspred_cols = boosting_modelspred = initial_model.predict# calculate residual predictions from each model and add models = boosting_modelslearning_rate = boosting_modelsfor model in models: temp_resid_preds = model.predictpred += learning_rate*temp_resid_preds if chart: plt.scatterplt.showrmse = np.sqrt) return pred, rmse Sweet, let’s make a model on the same diabetes dataset that we used in the bagging section. We’ll do a quick grid searchto tune our three parameters and then we’ll train the final model using the boost_resid_correction function. # tune parameters with grid search n_trees =learning_rates =max_depths = my_list = list) # Create a dictionary to hold test RMSE for each 'square' in grid perf_dict = {} for tree in n_trees: for learning_rate in learning_rates: for max_depth in max_depths: temp_boosted_model = boost_resid_correctiontemp_boosted_model= 'target' preds, rmse = boost_resid_correction_predictdict_key = '_'.joinfor x in) perf_dict= rmse min_key = minprintAnd our winner is — 50 trees, a learning rate of 0.1 and a max depth of 1! Let’s take a look and see how our predictions did. Tuned boosting actuals vs. residuals – image by author While our boosting ensemble model seems to capture the trend reasonably well, we can see off the bat that it isn’t predicting as well as the bagging model. We could probably spend more time tuning – but it could also be the case that the bagging approach fits this specific data better. With that said, we’ve now earned an understanding of bagging and boosting – let’s compare them in the next section! Bagging vs. Boosting – understanding the differences We’ve covered bagging and boosting separately, the table below brings all the information we’ve covered to concisely compare the approaches: image by author Note: In this article, we wrote our own bagging and boosting code for educational purposes. In practice you will just use the excellent code that is available in Python packages or other software. Also, people rarely use ‘pure’ bagging or boosting – it is much more common to use more advanced algorithms that modify the plain vanilla bagging and boosting to improve performance. Wrapping it up Bagging and boosting are powerful and practical ways to improve weak learners like the humble but flexible decision tree. Both approaches use the power of ensembling to address different problems – bagging for variance, boosting for bias. In practice, pre-packaged code is almost always used to train more advanced machine learning models that use the main ideas of bagging and boosting but, expand on them with multiple improvements. I hope that this has been helpful and interesting – happy modeling! Dataset is originally from the National Institute of Diabetes and Digestive and Kidney Diseases and is distributed under the public domain license for use without restriction. The post Strength in Numbers: Ensembling Models with Bagging and Boosting appeared first on Towards Data Science. #strength #numbers #ensembling #models #with