When Leta Sobierajski enrolled in college, she already knew what she was meant to do, and she didn’t settle for anything less. “When I went to school for graphic design, I really didn’t have a backup plan – it was this, or nothing,” she says. “My work is a constantly evolving practice, and from the beginning, I have always convinced myself that if I put in the time and experimentation, I would grow and evolve.” After graduation, Sobierajski took on a range of projects, which included animation, print, and branding elements. She collaborated with corporate clients, but realized that she wouldn’t feel comfortable following anyone else’s rules in a 9-to-5 environment. Leta Sobierajski (standing) and Wade Jeffree (on ladder) \\\ Photo: Matt Dutile Sobierajski eventually decided to team up with fellow artist and kindred spirit Wade Jeffree. In 2016 they launched their Brooklyn-based studio, Wade and Leta. The duo, who share a taste for quirky aesthetics, produces sculpture, installations, or anything else they can dream up. Never static in thinking or method, they are constantly searching for another medium to try that will complement their shared vision of the moment. The pair is currently interested in permanency, and they want to utilize more metal, a strong material that will stand the test of time. Small architectural pieces are also on tap, and on a grander scale, they’d like to focus on a park or communal area that everyone can enjoy. With so many ideas swirling around, Sobierajski will record a concept in at least three different ways so that she’s sure to unearth it at a later date. “In some ways, I like to think I’m impeccably organized, as I have countless spreadsheets tracking our work, our lives, and our well-being,” she explains. “The reality is that I am great at over-complicating situations with my intensified list-making and note-taking. The only thing to do is to trust the process.” Today, Leta Sobierajski joins us for Friday Five! Photo: Melitta Baumeister and Michał Plata 1. Melitta Baumeister and Michał Plata The work of Melitta Baumeister and Michał Plata has been a constant inspiration to me for their innovative, artful, and architectural silhouettes. By a practice of draping and arduous pattern-making, the garments that they develop season after season feel like they could be designed for existence in another universe. I’m a person who likes to dress up for anything when I’m not in the studio, and every time I opt to wear one of their looks, I feel like I can take on the world. The best part about their pieces is that they’re extremely functional, so whether I need to hop on a bicycle or show up at an opening, I’m still able to make a statement – these garments even have the ability to strike up conversations on their own. Photo: Wade and Leta 2. Pandas! I was recently in Chengdu to launch a new project and we took half the day to visit the Chengdu Research Base of Giant Pandas and I am a new panda convert. Yes, they’re docile and cute, but their lifestyles are utterly chill and deeply enviable for us adults with responsibilities. Giant pandas primarily eat bamboo and can consume 20-40 kilograms per day. When they’re not doing that, they’re sleeping. When we visited, many could be seen reclining on their backs, feasting on some of the finest bamboo they could select within arm’s reach. While not necessarily playful in appearance, they do seem quite cheeky in their agendas and will do as little as they can to make the most of their meals. It felt like I was watching a mirrored image of myself on a Sunday afternoon while trying to make the most of my last hours of the weekend. Photo: Courtesy of Aoiro 3. Aoiro I’m not really a candle person (I forget to light it, and then I forget it’s lit, and then I panic when it’s been lit for too long) but I love the luxurious subtlety of a fragrant space. It’s an intangible feeling that really can only be experienced in the present. Some of the best people to create these fragrances, in my opinion, are Shizuko and Manuel, the masterminds behind Aoiro, a Japanese and Austrian duo who have developed a keen sense for embodying the fragrances of some of the most intriguing and captivating olfactory atmospheres – earthy forest floors with crackling pine needles, blue cypress tickling the moon in an indigo sky, and rainfall on a spirited Japanese island. Despite living in an urban city, Aoiro’s olfactory design is capable of transporting me to the deepest forests of misty Yakushima island. Photo: Wade and Leta 4. Takuro Kuwata A few months ago, I saw the work of Japanese ceramicist Takuro Kuwata at an exhibition at Salon94 and have been having trouble getting it out of my head. Kuwata’s work exemplifies someone who has worked with a medium so much to completely use the medium as a medium – if that makes sense. His ability to manipulate clay and glaze and use it to create gravity-defying effects within the kiln are exceptionally mysterious to me and feel like they could only be accomplished with years and years of experimentation with the material. I’m equally impressed seeing how he’s grown his work with scale, juxtaposing it with familiar iconography like the fuzzy peach, but sculpting it from materials like bronze. Photo: Wade and Leta 5. The Site of Reversible Destiny, a park built by artists Arakawa and Gins, in Yoro Japan The park is a testament to their career as writers, architects, and their idea of reversible destiny, which in its most extreme form, eliminates death. For all that are willing to listen, Arakawa and Gins’ Reversible Destiny mentality aims to make our lives a little more youthful by encouraging us to reevaluate our relationship with architecture and our surroundings. The intention of “reversible destiny” is not to prolong death, postpone it, grow older alongside it, but to entirely not acknowledge and surpass it. Wade (my partner) and I have spent the last ten years traveling to as many of their remaining sites as possible to further understand this notion of creating spaces to extend our lives and question how conventional living spaces can become detrimental to our longevity.   Works by Wade and Leta: Photo: Wade and Leta and Matt Alexander Now You See Me is a large-scale installation in the heart of Shoreditch, London, that explores the relationship between positive and negative space through bold color, geometry, and light. Simple, familiar shapes are embedded within monolithic forms, creating a layered visual experience that shifts throughout the day. As sunlight passes through the structures, shadows and silhouettes stretch and connect, forming dynamic compositions on the surrounding concrete. Photo: Wade and Leta and John Wylie Paint Your Own Path is series of five towering sculptures, ranging from 10 to 15 feet tall, invites viewers to explore balance, tension, and perspective through bold color and form. Inspired by the delicate, often precarious act of stacking objects, the sculptures appear as if they might topple – yet each one holds steady, challenging perceptions of stability. Created in partnership with the Corolla Cross, the installation transforms its environment into a pop-colored landscape. Photo: Millenia Walk and Outer Edit, Eurthe Studio Monument to Movement is a 14-meter-tall kinetic sculpture that celebrates the spirit of the holiday season through rhythm, motion, and color. Rising skyward in layered compositions, the work symbolizes collective joy, renewal, and the shared energy of celebrations that span cultures and traditions. Powered by motors and constructed from metal beams and cardboard forms, the sculpture continuously shifts, inviting viewers to reflect on the passage of time and the cycles that connect us all. Photo: Wade and Leta and Erika Hara, Piotr Maslanka, and Jeremy Renault Falling Into Place is a vibrant rooftop installation at Ginza Six that explores themes of alignment, adaptability, and perspective. Six colorful structures – each with a void like a missing puzzle piece – serve as spaces for reflection, inviting visitors to consider their place within a greater whole. Rather than focusing on absence, the design transforms emptiness into opportunity, encouraging people to embrace spontaneity and the unfolding nature of life. Playful yet contemplative, the work emphasizes that only through connection and participation can the full picture come into view. Photo: Wade and Leta and Erika Hara, Piotr Maslanka, and Jeremy Renault Photo: Wade and Leta Stop, Listen, Look is a 7-meter-tall interactive artwork atop IFS Chengdu that captures the vibrant rhythm of the city through movement, sound, and form. Blending motorized and wind-powered elements with seesaws and sound modulation, it invites people of all ages to engage, play, and reflect. Inspired by Chengdu’s balance of tradition and modernity, the piece incorporates circular motifs from local symbolism alongside bold, geometric forms to create a dialogue between past and present. With light, motion, and community at its core, the work invites visitors to connect with the city – and each other – through shared interaction. The Cloud is a permanent sculptural kiosk in Burlington, Vermont’s historic City Hall Park, created in collaboration with Brooklyn-based Studio RENZ+OEI. Designed to reinterpret the ephemeral nature of clouds through architecture, it blends art, air, and imagination into a light, fluid structure that defies traditional rigidity. Originally born from a creative exchange between longtime friends and collaborators, the design challenges expectations of permanence by embodying movement and openness. Now home to a local food vendor, The Cloud brings a playful, uplifting presence to the park, inviting reflection and interaction rain or shine..

Local Large Language Models can convert massive DataFrames to presentable Markdown reports — here's how. The post LLMs + Pandas: How I Use Generative AI to Generate Pandas DataFrame Summaries appeared first on Towards Data Science.

Adapting the iconic, block-based design aesthetic of Mojang’s beloved Minecraft videogame into the hit feature film comedy adventure, The Minecraft Movie, posed an enormous number of hurdles for director Jared Hess and Oscar-winning Production VFX Supervisor Dan Lemmon. Tasked with helping translate the iconic pixelated world into something cinematically engaging, while remaining true to its visual DNA, was Wētā FX, who delivered 450 VFX shots on the film. And two of their key leads on the film were VFX Supervisor Sheldon Stopsack and Animation Supervisor Kevin Estey.  But the shot count merely scratches the surface of the extensive work the studio performed. Wētā led the design and creation of The Overworld, 64 unique terrains spanning deserts, lush forests, oceans, and mountain ranges, all combined into one continuous environment, assets that were also shared with Digital Domain for their work on the 3rd act battle. Wētā also handled extensive work on the lava-filled hellscape of The Nether that involved Unreal Engine for early representations used in previs, scene scouting, and onset during principal photography, before refining the environment during post-production. They also dressed The Nether with lava, fire, and torches, along with atmospherics and particulate like smoke, ash, and embers. But wait… there’s more! The studio’s Art Department, working closely with Hess, co-created the look and feel of all digital characters in the film. For Malgosha’s henchmen, the Piglins, Wētā designed and created 12 different variants, all with individual characteristics and personalities. They also designed sheep, bees, pandas, zombies, skeletons, and lovable wolf Dennis. Many of these characters were provided to other vendors for their work on the film. Needless to say, the studio truly became a “Master Builder” on the show. The film is based on the hugely popular game Minecraft, first released by Sweden’s Mojang Studios in 2011 and purchased by Microsoft for $2.5 billion in 2014, which immerses players in a low-res, pixelated “sandbox” simulation where they can use blocks to build entire worlds.  Here's the final trailer: In a far-ranging interview, Stopsack and Estey shared with AWN a peek into their creative process, from early design exploration to creation of an intricate practical cloak for Malgosha and the use of Unreal Engine for previs, postvis, and real-time onset visualization. Dan Sarto: The film is filled with distinct settings and characters sporting various “block” styled features. Can you share some of the work you did on the environments, character design, and character animation? Sheldon Stopsack: There's, there's so much to talk about and truth to be told, if you were to touch on everything, we would probably need to spend the whole day together.  Kevin Estey: Sheldon and I realized that when we talk about the film, either amongst ourselves or with someone else, we could just keep going, there are so many stories to tell. DS: Well, start with The Overworld and The Nether. How did the design process begin? What did you have to work with? SS: Visual effects is a tricky business, you know. It's always difficult. Always challenging. However, Minecraft stood out to us as not your usual quote unquote standard visual effects project, even though as you know, there is no standard visual effects project because they're all somehow different. They all come with their own creative ideas, inspirations, and challenges. But Minecraft, right from the get-go, was different, simply by the fact that when you first consider the idea of making such a live-action movie, you instantly ask yourself, “How do we make this work? How do we combine these two inherently very, very different but unique worlds?” That was everyone’s number one question. How do we land this? Where do we land this? And I don't think that any of us really had an answer, including our clients, Dan Lemmon [Production VFX Supervisor] and Jared Hess [the film’s director]. Everyone was really open for this journey. That's compelling for us, to get out of our comfort zone. It makes you nervous because there are no real obvious answers. KE: Early on, we seemed to thrive off these kinds of scary creative challenges. There were lots of question marks. We had many moments when we were trying to figure out character designs. We had a template from the game, but it was an incredibly vague, low-resolution template. And there were so many ways that we could go. But that design discovery throughout the project was really satisfying.  DS: Game adaptations are never simple. There usually isn’t much in the way of story. But with Minecraft, from a visual standpoint, how did you translate low res, block-styled characters into something entertaining that could sustain a 100-minute feature film? SS: Everything was a question mark. Using the lava that you see in The Nether as one example, we had beautiful concept art for all our environments, The Overworld and The Nether, but those concepts only really took you this far. They didn’t represent the block shapes or give you a clear answer of like how realistic some of those materials, shapes and structures would be. How organic would we go? All of this needed to be explored. For the lava, we had stylized concept pieces, with block shaped viscosity as it flowed down. But we spent months with our effects team, and Dan and Jared, just riffing on ideas. We came full circle, with the lava ending up being more realistic, a naturally viscous liquid based on real physics. And the same goes with the waterfall that you see in the Overworld.  The question is, how far do we take things into the true Minecraft representation of things? How much do we scale back a little bit and ground ourselves in reality, with effects we’re quite comfortable producing as a company? There's always a tradeoff to find that balance of how best to combine what’s been filmed, the practical sets and live-action performances, with effects. Where’s the sweet spot? What's the level of abstraction? What's honest to the game? As much as some call Minecraft a simple game, it isn't simple, right? It's incredibly complex. It's got a set of rules and logic to the world building process within the game that we had to learn, adapt, and honor in many ways. When our misfits first arrive and we have these big vistas and establishing shots, when you really look at it, you, you recognize a lot of the things that we tried to adapt from the game. There are different biomes, like the Badlands, which is very sand stoney; there's the Woodlands, which is a lush environment with cherry blossom trees; you’ve got the snow biome with big mountains in the background. Our intent was to honor the game. KE: I took a big cue from a lot of the early designs, and particularly the approach that Jared liked for the characters and to the design in general, which was maintaining the stylized, blocky aesthetic, but covering them in realistic flesh, fur, things that were going to make them appear as real as possible despite the absolutely unreal designs of their bodies. And so essentially, it was squared skeleton… squarish bones with flesh and realistic fur laid over top. We tried various things, all extremely stylized. The Creepers are a good example. We tried all kinds of ways for them to explode. Sheldon found a great reference for a cat coughing up a hairball. He was nice to censor the worst part of it, but those undulations in the chest and ribcage… Jared spoke of the Creepers being basically tragic characters that only wanted to be loved, to just be close to you. But sadly, whenever they did, they’d explode. So, we experimented with a lot of different motions of how they’d explode. DS: Talk about the process of determining how these characters would move. None seem to have remotely realistic proportions in their limbs, bodies, or head size. KE: There were a couple things that Jared always seemed to be chasing. One was just something that would make him laugh. Of course, it had to sit within the bounds of how a zombie might move, or a skeleton might move, as we were interpreting the game. But the main thing was just, was it fun and funny? I still remember one of the earliest gags they came up with in mocap sessions, even before I even joined the show, was how the zombies get up after they fall over. It was sort of like a tripod, where its face and feet were planted and its butt shoots up in the air. After a lot of experimentation, we came up with basic personality types for each character. There were 12 different types of Piglins. The zombies were essentially like you're coming home from the pub after a few too many pints and you're just trying to get in the door, but you can't find your keys. Loose, slightly inebriated movement. The best movement we found for the skeletons was essentially like an old man with rigid limbs and lack of ligaments that was chasing kids off his lawn. And so, we created this kind of bible of performance types that really helped guide performers on the mocap stage and animators later on. SS: A lot of our exploration didn’t stick. But Jared was the expert in all of this. He always came up with some quirky last-minute idea.  KE: My favorite from Jared came in the middle of one mocap shoot. He walked up to me and said he had this stupid idea. I said OK, go on. He said, what if Malgosha had these two little pigs next to her, like Catholic alter boys [the thurifers], swinging incense [a thurible]. Can we do that? I talked to our stage manager, and we quickly put together a temporary prop for the incense burners. And we got two performers who just stood there. What are they going to do? Jared said, “Nothing. Just stand there and swing. I think it would look funny.” So, that’s what we did.  We dubbed them the Priesty Boys. And they are there throughout the film. That was amazing about Jared. He was always like, let's just try it, see if it works. Otherwise ditch it. DS: Tell me about your work on Malgosha. And I also want to discuss your use of Unreal Engine and the previs and postvis work.  SS: For Malgosha as a character, our art department did a phenomenal job finding the character design at the concept phase. But it was a collective effort. So many contributors were involved in her making. And I'm not just talking about the digital artists here on our side. It was a joint venture of different people having different explorations and experiments. It started off with the concept work as a foundation, which we mocked up with 3D sketches before building a model. But with Malgosha, we also had the costume department on the production side building this elaborate cloak. Remember, that cloak kind of makes 80, 85% of her appearance. It's almost like a character in itself, the way we utilized it. And the costume department built this beautiful, elaborate, incredibly intricate, practical version of it that we intended to use on set for the performer to wear. It ended up being too impractical because it was too heavy. But it was beautiful. So, while we didn't really use it on set, it gave us something physically to kind of incorporate into our digital version. KE: Alan Henry is the motion performer who portrayed her on set and on the mocap stage. I've known him for close to 15 years. I started working with him on The Hobbit films. He was a stunt performer who eventually rolled into doing motion capture with us on The Hobbit. He’s an incredible actor and absolutely hilarious and can adapt to any sort of situation. He’s so improvisational. He came up with an approach to Malgosha very quickly. Added a limp so that she felt decrepit, leaning on the staff, adding her other arm as kind of like a gimp arm that she would point and gesture with.   Even though she’s a blocky character, her anatomy is very much a biped, with rounder limbs than the other Piglins. She's got hooves, is somewhat squarish, and her much more bulky mass in the middle was easier to manipulate and move around. Because she would have to battle with Steve in the end, she had to have a level of agility that even some of the Piglins didn't have. DS: Did Unreal Engine come into play with her?  SS: Unreal was used all the way through the project. Dan Lemmon and his team early on set up their own virtual art department to build representations of the Overworld and the Nether within the context of Unreal. We and Sony Imageworks tried to provide recreations of these environments that were then used within Unreal to previsualize what was happening on set during shooting of principal photography. And that's where our mocap and on-set teams were coming into play. Effects provided what we called the Nudge Cam. It was a system to do real-time tracking using a stereo pair of Basler computer vision cameras that were mounted onto the sides of the principal camera. We provided the live tracking that was then composited in real time with the Unreal Engine content that all the vendors had provided. It was a great way of utilizing Unreal to give the camera operators or DOP, even Jared, a good sense of what we would actually shoot. It gave everyone a little bit of context for the look and feel of what you could actually expect from these scenes.  Because we started this journey with Unreal having onset in mind, we internally decided, look, let's take this further. Let's take this into post-production as well. What would it take to utilize Unreal for shot creation? And it was really exclusively used on the Nether environment. I don’t want to say we used it for matte painting replacement. We used it more for say, let's build this extended environment in Unreal. Not only use it as a render engine with this reasonably fast turnaround but also use it for what it's good at: authoring things, quickly changing things, moving columns around, manipulating things, dressing them, lighting them, and rendering them. It became sort of a tool that we used in place of a traditional matte painting for the extended environments. KE: Another thing worth mentioning is we were able to utilize it on our mocap stage as well during the two-week shoot with Jared and crew. When we shoot on the mocap stage, we get a very simple sort of gray shaded diagnostic grid. You have your single-color characters that sometimes are textured, but they’re fairly simple without any context of environment. Our special projects team was able to port what we usually see in Giant, the software we use on the mocap stage, into Unreal, which gave us these beautifully lit environments with interactive fire and atmosphere. And Jared and the team could see their movie for the first time in a rough, but still very beautiful rough state. That was invaluable. DS: If you had to key on anything, what would say with the biggest challenges for your teams on the film? You're laughing. I can hear you thinking, “Do we have an hour?”  KE: Where do you begin?  SS: Exactly. It's so hard to really single one out. And I struggle with that question every time I've been asked that question. KE: I’ll start.  I've got a very simple practical answer and then a larger one, something that was new to us, kind of similar to what we were just talking about. The simple practical one is the Piglins square feet with no ankles. It was very tough to make them walk realistically. Think of the leg of a chair. How do you make that roll and bank and bend because there is no joint? There are a lot of Piglins walking on surfaces and it was a very difficult conundrum to solve. It took a lot of hard work from our motion edit team and our animation team to get those things walking realistically. You know, it’s doing that simple thing that you don't usually pay attention to. So that was one reasonably big challenge that is often literally buried in the shadows. The bigger one was something that was new to me. We often do a lot of our previs and postvis in-house and then finish the shots. And just because of circumstances and capacity, we did the postvis for the entire final battle, but we ended up sharing the sequence with Digital Domain, who did an amazing job completing some of the stuff on the Battlefield we did post on. For me personally, I've never experienced not finishing what I started. But it was also really rewarding to see how well the work we had put in was honored by DD when they took it over.   SS: I think the biggest challenge and the biggest achievement that I'm most proud of is really ending up with something that was well received by the wider audience. Of creating these two worlds, this sort of abstract adaptation of the Minecraft game and combining it with live-action. That was the achievement for me. That was the biggest challenge. We were all nervous from day one. And we continued to be nervous up until the day the movie came out. None of us really knew how it ultimately would be received. The fact that it came together and was so well received is a testament to everyone doing a fantastic job. And that's what I'm incredibly proud of. Dan Sarto is Publisher and Editor-in-Chief of Animation World Network.

Real-world data is often costly, messy, and limited by privacy rules. Synthetic data offers a solution—and it’s already widely used: LLMs train on AI-generated text Fraud systems simulate edge cases Vision models pretrain on fake images SDV (Synthetic Data Vault) is an open-source Python library that generates realistic tabular data using machine learning. It learns patterns from real data and creates high-quality synthetic data for safe sharing, testing, and model training. In this tutorial, we’ll use SDV to generate synthetic data step by step. pip install sdv We will first install the sdv library: from sdv.io.local import CSVHandler connector = CSVHandler() FOLDER_NAME = '.' # If the data is in the same directory data = connector.read(folder_name=FOLDER_NAME) salesDf = data['data'] Next, we import the necessary module and connect to our local folder containing the dataset files. This reads the CSV files from the specified folder and stores them as pandas DataFrames. In this case, we access the main dataset using data[‘data’]. from sdv.metadata import Metadata metadata = Metadata.load_from_json('metadata.json') We now import the metadata for our dataset. This metadata is stored in a JSON file and tells SDV how to interpret your data. It includes: The table name The primary key The data type of each column (e.g., categorical, numerical, datetime, etc.) Optional column formats like datetime patterns or ID patterns Table relationships (for multi-table setups) Here is a sample metadata.json format: { "METADATA_SPEC_VERSION": "V1", "tables": { "your_table_name": { "primary_key": "your_primary_key_column", "columns": { "your_primary_key_column": { "sdtype": "id", "regex_format": "T[0-9]{6}" }, "date_column": { "sdtype": "datetime", "datetime_format": "%d-%m-%Y" }, "category_column": { "sdtype": "categorical" }, "numeric_column": { "sdtype": "numerical" } }, "column_relationships": [] } } } from sdv.metadata import Metadata metadata = Metadata.detect_from_dataframes(data) Alternatively, we can use the SDV library to automatically infer the metadata. However, the results may not always be accurate or complete, so you might need to review and update it if there are any discrepancies. from sdv.single_table import GaussianCopulaSynthesizer synthesizer = GaussianCopulaSynthesizer(metadata) synthesizer.fit(data=salesDf) synthetic_data = synthesizer.sample(num_rows=10000) With the metadata and original dataset ready, we can now use SDV to train a model and generate synthetic data. The model learns the structure and patterns in your real dataset and uses that knowledge to create synthetic records. You can control how many rows to generate using the num_rows argument. from sdv.evaluation.single_table import evaluate_quality quality_report = evaluate_quality( salesDf, synthetic_data, metadata) The SDV library also provides tools to evaluate the quality of your synthetic data by comparing it to the original dataset. A great place to start is by generating a quality report You can also visualize how the synthetic data compares to the real data using SDV’s built-in plotting tools. For example, import get_column_plot from sdv.evaluation.single_table to create comparison plots for specific columns: from sdv.evaluation.single_table import get_column_plot fig = get_column_plot( real_data=salesDf, synthetic_data=synthetic_data, column_name='Sales', metadata=metadata ) fig.show() We can observe that the distribution of the ‘Sales’ column in the real and synthetic data is very similar. To explore further, we can use matplotlib to create more detailed comparisons—such as visualizing the average monthly sales trends across both datasets. import pandas as pd import matplotlib.pyplot as plt # Ensure 'Date' columns are datetime salesDf['Date'] = pd.to_datetime(salesDf['Date'], format='%d-%m-%Y') synthetic_data['Date'] = pd.to_datetime(synthetic_data['Date'], format='%d-%m-%Y') # Extract 'Month' as year-month string salesDf['Month'] = salesDf['Date'].dt.to_period('M').astype(str) synthetic_data['Month'] = synthetic_data['Date'].dt.to_period('M').astype(str) # Group by 'Month' and calculate average sales actual_avg_monthly = salesDf.groupby('Month')['Sales'].mean().rename('Actual Average Sales') synthetic_avg_monthly = synthetic_data.groupby('Month')['Sales'].mean().rename('Synthetic Average Sales') # Merge the two series into a DataFrame avg_monthly_comparison = pd.concat([actual_avg_monthly, synthetic_avg_monthly], axis=1).fillna(0) # Plot plt.figure(figsize=(10, 6)) plt.plot(avg_monthly_comparison.index, avg_monthly_comparison['Actual Average Sales'], label='Actual Average Sales', marker='o') plt.plot(avg_monthly_comparison.index, avg_monthly_comparison['Synthetic Average Sales'], label='Synthetic Average Sales', marker='o') plt.title('Average Monthly Sales Comparison: Actual vs Synthetic') plt.xlabel('Month') plt.ylabel('Average Sales') plt.xticks(rotation=45) plt.grid(True) plt.legend() plt.ylim(bottom=0) # y-axis starts at 0 plt.tight_layout() plt.show() This chart also shows that the average monthly sales in both datasets are very similar, with only minimal differences. In this tutorial, we demonstrated how to prepare your data and metadata for synthetic data generation using the SDV library. By training a model on your original dataset, SDV can create high-quality synthetic data that closely mirrors the real data’s patterns and distributions. We also explored how to evaluate and visualize the synthetic data, confirming that key metrics like sales distributions and monthly trends remain consistent. Synthetic data offers a powerful way to overcome privacy and availability challenges while enabling robust data analysis and machine learning workflows. Check out the Notebook on GitHub. All credit for this research goes to the researchers of this project. Also, feel free to follow us on Twitter and don’t forget to join our 95k+ ML SubReddit and Subscribe to our Newsletter. Arham IslamI am a Civil Engineering Graduate (2022) from Jamia Millia Islamia, New Delhi, and I have a keen interest in Data Science, especially Neural Networks and their application in various areas.Arham Islamhttps://www.marktechpost.com/author/arhamislam/Step-by-Step Guide to Create an AI agent with Google ADKArham Islamhttps://www.marktechpost.com/author/arhamislam/Implementing an LLM Agent with Tool Access Using MCP-UseArham Islamhttps://www.marktechpost.com/author/arhamislam/Implementing an AgentQL Model Context Protocol (MCP) ServerArham Islamhttps://www.marktechpost.com/author/arhamislam/Implementing An Airbnb and Excel MCP Server

Who gets hired as an NBA coach? How long does a typical coach last? And does their coaching background play any part in predicting success? This analysis was inspired by several key theories. First, there has been a common criticism among casual NBA fans that teams overly prefer hiring candidates with previous NBA head coaches experience. Consequently, this analysis aims to answer two related questions. First, is it true that NBA teams frequently re-hire candidates with previous head coaching experience? And second, is there any evidence that these candidates under-perform relative to other candidates? The second theory is that internal candidates (though infrequently hired) are often more successful than external candidates. This theory was derived from a pair of anecdotes. Two of the most successful coaches in NBA history, Gregg Popovich of San Antonio and Erik Spoelstra of Miami, were both internal hires. However, rigorous quantitative evidence is needed to test if this relationship holds over a larger sample. This analysis aims to explore these questions, and provide the code to reproduce the analysis in Python. The Data The code (contained in a Jupyter notebook) and dataset for this project are available on Github here. The analysis was performed using Python in Google Colaboratory.  A prerequisite to this analysis was determining a way to measure coaching success quantitatively. I decided on a simple idea: the success of a coach would be best measured by the length of their tenure in that job. Tenure best represents the differing expectations that might be placed on a coach. A coach hired to a contending team would be expected to win games and generate deep playoff runs. A coach hired to a rebuilding team might be judged on the development of younger players and their ability to build a strong culture. If a coach meets expectations (whatever those may be), the team will keep them around. Since there was no existing dataset with all of the required data, I collected the data myself from Wikipedia. I recorded every off-season coaching change from 1990 through 2021. Since the primary outcome variable is tenure, in-season coaching changes were excluded since these coaches often carried an “interim” tag—meaning they were intended to be temporary until a permanent replacement could be found. In addition, the following variables were collected: VariableDefinitionTeamThe NBA team the coach was hired forYearThe year the coach was hiredCoachThe name of the coachInternal?An indicator if the coach was internal or not—meaning they worked for the organization in some capacity immediately prior to being hired as head coachTypeThe background of the coach. Categories are Previous HC (prior NBA head coaching experience), Previous AC (prior NBA assistant coaching experience, but no head coaching experience), College (head coach of a college team), Player (a former NBA player with no coaching experience), Management (someone with front office experience but no coaching experience), and Foreign (someone coaching outside of North America with no NBA coaching experience).YearsThe number of years a coach was employed in the role. For coaches fired mid-season, the value was counted as 0.5. First, the dataset is imported from its location in Google Drive. I also convert ‘Internal?’ into a dummy variable, replacing “Yes” with 1 and “No” with 0. from google.colab import drive drive.mount('/content/drive') import pandas as pd pd.set_option('display.max_columns', None) #Bring in the dataset coach = pd.read_csv('/content/drive/MyDrive/Python_Files/Coaches.csv', on_bad_lines = 'skip').iloc[:,0:6] coach['Internal'] = coach['Internal?'].map(dict(Yes=1, No=0)) coach This prints a preview of what the dataset looks like: In total, the dataset contains 221 coaching hires over this time.  Descriptive Statistics First, basic summary Statistics are calculated and visualized to determine the backgrounds of NBA head coaches. #Create chart of coaching background import matplotlib.pyplot as plt #Count number of coaches per category counts = coach['Type'].value_counts() #Create chart plt.bar(counts.index, counts.values, color = 'blue', edgecolor = 'black') plt.title('Where Do NBA Coaches Come From?') plt.figtext(0.76, -0.1, "Made by Brayden Gerrard", ha="center") plt.xticks(rotation = 45) plt.ylabel('Number of Coaches') plt.gca().spines['top'].set_visible(False) plt.gca().spines['right'].set_visible(False) for i, value in enumerate(counts.values): plt.text(i, value + 1, str(round((value/sum(counts.values))*100,1)) + '%' + ' (' + str(value) + ')', ha='center', fontsize=9) plt.savefig('coachtype.png', bbox_inches = 'tight') print(str(round(((coach['Internal'] == 1).sum()/len(coach))*100,1)) + " percent of coaches are internal.") Over half of coaching hires previously served as an NBA head coach, and nearly 90% had NBA coaching experience of some kind. This answers the first question posed—NBA teams show a strong preference for experienced head coaches. If you get hired once as an NBA coach, your odds of being hired again are much higher. Additionally, 13.6% of hires are internal, confirming that teams do not frequently hire from their own ranks. Second, I will explore the typical tenure of an NBA head coach. This can be visualized using a histogram. #Create histogram plt.hist(coach['Years'], bins =12, edgecolor = 'black', color = 'blue') plt.title('Distribution of Coaching Tenure') plt.figtext(0.76, 0, "Made by Brayden Gerrard", ha="center") plt.annotate('Erik Spoelstra (MIA)', xy=(16.4, 2), xytext=(14 + 1, 15), arrowprops=dict(facecolor='black', shrink=0.1), fontsize=9, color='black') plt.gca().spines['top'].set_visible(False) plt.gca().spines['right'].set_visible(False) plt.savefig('tenurehist.png', bbox_inches = 'tight') plt.show() coach.sort_values('Years', ascending = False) #Calculate some stats with the data import numpy as np print(str(np.median(coach['Years'])) + " years is the median coaching tenure length.") print(str(round(((coach['Years'] <= 5).sum()/len(coach))*100,1)) + " percent of coaches last five years or less.") print(str(round((coach['Years'] <= 1).sum()/len(coach)*100,1)) + " percent of coaches last a year or less.") Using tenure as an indicator of success, the the data clearly shows that the large majority of coaches are unsuccessful. The median tenure is just 2.5 seasons. 18.1% of coaches last a single season or less, and barely 10% of coaches last more than 5 seasons. This can also be viewed as a survival analysis plot to see the drop-off at various points in time: #Survival analysis import matplotlib.ticker as mtick lst = np.arange(0,18,0.5) surv = pd.DataFrame(lst, columns = ['Period']) surv['Number'] = np.nan for i in range(0,len(surv)): surv.iloc[i,1] = (coach['Years'] >= surv.iloc[i,0]).sum()/len(coach) plt.step(surv['Period'],surv['Number']) plt.title('NBA Coach Survival Rate') plt.xlabel('Coaching Tenure (Years)') plt.figtext(0.76, -0.05, "Made by Brayden Gerrard", ha="center") plt.gca().yaxis.set_major_formatter(mtick.PercentFormatter(1)) plt.gca().spines['top'].set_visible(False) plt.gca().spines['right'].set_visible(False) plt.savefig('coachsurvival.png', bbox_inches = 'tight') plt.show Lastly, a box plot can be generated to see if there are any obvious differences in tenure based on coaching type. Boxplots also display outliers for each group. #Create a boxplot import seaborn as sns sns.boxplot(data=coach, x='Type', y='Years') plt.title('Coaching Tenure by Coach Type') plt.gca().spines['top'].set_visible(False) plt.gca().spines['right'].set_visible(False) plt.xlabel('') plt.xticks(rotation = 30, ha = 'right') plt.figtext(0.76, -0.1, "Made by Brayden Gerrard", ha="center") plt.savefig('coachtypeboxplot.png', bbox_inches = 'tight') plt.show There are some differences between the groups. Aside from management hires (which have a sample of just six), previous head coaches have the longest average tenure at 3.3 years. However, since many of the groups have small sample sizes, we need to use more advanced techniques to test if the differences are statistically significant. Statistical Analysis First, to test if either Type or Internal has a statistically significant difference among the group means, we can use ANOVA: #ANOVA import statsmodels.api as sm from statsmodels.formula.api import ols am = ols('Years ~ C(Type) + C(Internal)', data=coach).fit() anova_table = sm.stats.anova_lm(am, typ=2) print(anova_table) The results show high p-values and low F-stats—indicating no evidence of statistically significant difference in means. Thus, the initial conclusion is that there is no evidence NBA teams are under-valuing internal candidates or over-valuing previous head coaching experience as initially hypothesized.  However, there is a possible distortion when comparing group averages. NBA coaches are signed to contracts that typically run between three and five years. Teams typically have to pay out the remainder of the contract even if coaches are dismissed early for poor performance. A coach that lasts two years may be no worse than one that lasts three or four years—the difference could simply be attributable to the length and terms of the initial contract, which is in turn impacted by the desirability of the coach in the job market. Since coaches with prior experience are highly coveted, they may use that leverage to negotiate longer contracts and/or higher salaries, both of which could deter teams from terminating their employment too early. To account for this possibility, the outcome can be treated as binary rather than continuous. If a coach lasted more than 5 seasons, it is highly likely they completed at least their initial contract term and the team chose to extend or re-sign them. These coaches will be treated as successes, with those having a tenure of five years or less categorized as unsuccessful. To run this analysis, all coaching hires from 2020 and 2021 must be excluded, since they have not yet been able to eclipse 5 seasons. With a binary dependent variable, a logistic regression can be used to test if any of the variables predict coaching success. Internal and Type are both converted to dummy variables. Since previous head coaches represent the most common coaching hires, I set this as the “reference” category against which the others will be measured against. Additionally, the dataset contains just one foreign-hired coach (David Blatt) so this observation is dropped from the analysis. #Logistic regression coach3 = coach[coach['Year']<2020] coach3.loc[:, 'Success'] = np.where(coach3['Years'] > 5, 1, 0) coach_type_dummies = pd.get_dummies(coach3['Type'], prefix = 'Type').astype(int) coach_type_dummies.drop(columns=['Type_Previous HC'], inplace=True) coach3 = pd.concat([coach3, coach_type_dummies], axis = 1) #Drop foreign category / David Blatt since n = 1 coach3 = coach3.drop(columns=['Type_Foreign']) coach3 = coach3.loc[coach3['Coach'] != "David Blatt"] print(coach3['Success'].value_counts()) x = coach3[['Internal','Type_Management','Type_Player','Type_Previous AC', 'Type_College']] x = sm.add_constant(x) y = coach3['Success'] logm = sm.Logit(y,x) logm.r = logm.fit(maxiter=1000) print(logm.r.summary()) #Convert coefficients to odds ratio print(str(np.exp(-1.4715)) + "is the odds ratio for internal.") #Internal coefficient print(np.exp(1.0025)) #Management print(np.exp(-39.6956)) #Player print(np.exp(-0.3626)) #Previous AC print(np.exp(-0.6901)) #College Consistent with ANOVA results, none of the variables are statistically significant under any conventional threshold. However, closer examination of the coefficients tells an interesting story. The beta coefficients represent the change in the log-odds of the outcome. Since this is unintuitive to interpret, the coefficients can be converted to an Odds Ratio as follows: Internal has an odds ratio of 0.23—indicating that internal candidates are 77% less likely to be successful compared to external candidates. Management has an odds ratio of 2.725, indicating these candidates are 172.5% more likely to be successful. The odds ratios for players is effectively zero, 0.696 for previous assistant coaches, and 0.5 for college coaches. Since three out of four coaching type dummy variables have an odds ratio under one, this indicates that only management hires were more likely to be successful than previous head coaches. From a practical standpoint, these are large effect sizes. So why are the variables statistically insignificant? The cause is a limited sample size of successful coaches. Out of 202 coaches remaining in the sample, just 23 (11.4%) were successful. Regardless of the coach’s background, odds are low they last more than a few seasons. If we look at the one category able to outperform previous head coaches (management hires) specifically: # Filter to management manage = coach3[coach3['Type_Management'] == 1] print(manage['Success'].value_counts()) print(manage) The filtered dataset contains just 6 hires—of which just one (Steve Kerr with Golden State) is classified as a success. In other words, the entire effect was driven by a single successful observation. Thus, it would take a considerably larger sample size to be confident if differences exist. With a p-value of 0.202, the Internal variable comes the closest to statistical significance (though it still falls well short of a typical alpha of 0.05). Notably, however, the direction of the effect is actually the opposite of what was hypothesized—internal hires are less likely to be successful than external hires. Out of 26 internal hires, just one (Erik Spoelstra of Miami) met the criteria for success. Conclusion In conclusion, this analysis was able to draw several key conclusions: Regardless of background, being an NBA coach is typically a short-lived job. It’s rare for a coach to last more than a few seasons. The common wisdom that NBA teams strongly prefer to hire previous head coaches holds true. More than half of hires already had NBA head coaching experience. If teams don’t hire an experienced head coach, they’re likely to hire an NBA assistant coach. Hires outside of these two categories are especially uncommon. Though they are frequently hired, there is no evidence to suggest NBA teams overly prioritize previous head coaches. To the contrary, previous head coaches stay in the job longer on average and are more likely to outlast their initial contract term—though neither of these differences are statistically significant. Despite high-profile anecdotes, there is no evidence to suggest that internal hires are more successful than external hires either. Note: All images were created by the author unless otherwise credited. The post What Statistics Can Tell Us About NBA Coaches appeared first on Towards Data Science.