What do the Suez Canal, the Roman Goddess Libertas, and ancient Egyptian sculptures have in common? The Statue of Liberty. Surprising? Sure, but the connections make sense when you know the story as recounted by Columbia University psychologist Sheena Iyengar on a recent episode of Hidden Brain. The French artist Frédéric Bartholdi drew inspiration from Egyptian sculptures when he submitted a design for a sculpture that was going to be built at the Suez Canal. That plan for the Suez Canal sculpture fell through, leading Bartholdi and a friend to raise money to create a sculpture as a gift to the United States. Bartholdi designed the sculpture after studying the intricacies of the Roman Goddess Libertas, a significant female icon in the late 1800s. He also modeled the statue on Isabelle Boyer, who was 36 years old in 1878. Finally, Bartholdi incorporated his mother’s face into the proposed design. The result? The Statue of Liberty. Bartholdi’s unorthodox yet methodical approach yielded one of the most famous sculptures in the world. How did he do it? Did he let his mind run wild? Did he generate endless lists or draw hundreds of plans for each sculpture? Was he a 19th-century brainstorming advocate? The Problem “Yes,” would be the answer of many innovation experts today. From stand-ups to workshops and templates to whiteboards, getting the creative juices flowing often involves brainstorming along with the reminder that “there are no bad ideas” and “more ideas are better.” Practiced and repeated so often, this approach to creativity must work, right? Wrong, says Iyengar. Too many ideas hinder creativity because the human brain can only manage a few ideas at once. “Creativity requires you to have a bunch of pieces and to not only be able to have them in your memory bank in a way that you can kind of say what they are, but to be able to keep manipulating them in lots of different ways. And that means, you know, in order for your mind to be able to be facile enough to do that, it is going to need fewer pieces.”— Hidden Brain, “How to be more creative” Evidence for this view includes a study published by Anne-Laure Sellier of HEC Paris and Darren W. Dahl of British Columbia. The authors compared knitting and crafting in two experimental studies. The results suggested that restricting the number of materials and other creative inputs enhanced the creativity of study participants. The reason was the participants’ ability to enjoy the creative process more, which enhanced their creative output. A few years ago, I had a similar experience while planning a series of studies. As with any initiative, identifying the scope was key. The problem? Rather than choose from two or three well-defined options, the team discussed several themes at once and then piled on a series of ideas about the best format for presenting these themes: Lists, tables, graphs, images, and flowcharts. The results looked something like this. A messy whiteboard is not inherently bad. The question is whether brainstorming results like these block or enhance creativity. If the board above seems overwhelming, it’s worth considering a more structured process for creativity and idea generation. The Solution: Three Ways To Enhance Creativity Just as Bartholdi approached his designs methodically, designers today can benefit from limits and structure. In this article, I’ll shed light on three techniques that enhance creativity: Controlled curiosity Imposing constraints and making a plan Look to other domains Tip 1: Controlled Curiosity In today’s world, it’s easy to fall into the trap of believing that creativity comes from simply exposing yourself to a flood of information — scrolling endlessly, consuming random facts, and filling your mind with disconnected data points. It’s a trap because mindless absorption of information without understanding the purpose or deeper context won’t make you more creative. True creativity is fueled by curiosity, the drive to know more. Curiosity is powerful because it acts as an internal compass, guiding our search for knowledge with intention. When you’re curious, you don’t just passively take in information; you actively seek it with a purpose. You have a question in mind, a direction, a reason that shapes the way you explore. This sense of purpose transforms information from a chaotic influx of data into structured, meaningful insights that the brain can organize, categorize, and retrieve when needed. In my role as a user experience (UX) researcher, I recently needed to review 100+ internal and industry research papers to establish and understand what was already known about a specific subject. The challenge was how to sort, organize, and absorb this information without feeling overwhelmed. Was it better to leverage AI tools like Gemini or ChatGPT to summarize this body of knowledge? How reliable would these summaries be? Was it better to read the executive summaries and copy a few themes to include in a synopsis of all of these papers? What was the best way to organize this information? Which tool should I use to summarize and organize? Faced with a tight deadline and mounting stress, I paused to reassess. To avoid spiraling, I asked: What are the core objectives of this research review? I then defined three key goals: Extract three to five themes to present to several internal teams. Craft a research plan pegged to these themes. Leverage these themes to inform a series of screens that the design team would create to test with real users. With clearly defined objectives, I had a purpose. This purpose allowed me to channel my innate curiosity because I knew why I was wading through so much material and who would read and review the synthesis. Curiosity drove me to explore this large body of research, but purpose kept me focused. Curiosity is the drive to learn more. Creativity requires curiosity because, without this drive, designers and researchers are less likely to explore new ideas or new approaches to problem-solving. The good news is that research and design attract the naturally curious. The key lies in transforming curiosity into focused exploration. It’s less about the volume of information absorbed and more about the intent behind the inquiry, the depth of engagement, and the strategic application of acquired knowledge. Purposeful curiosity is the difference between drowning in a sea of knowledge and navigating it with mastery. Tip 2: Imposing Constraints And Making A Plan Just as purpose makes it easier to focus, constraint also contributes to creativity. Brainstorming 50 ideas might seem creative but can actually prove more distracting than energizing. Limiting the number of ideas is more productive. “Some people think that having constraints means they can’t be creative. The research shows that people are more creative when there are constraints.”— Dr. Susan Weinschenk, “The Role of Creativity in Design” The point is not to limit creativity and innovation but to nurture it with structure. Establishing constraints enhances creativity by focusing idea generation around a few key themes. Here are two ways to focus on idea generation: During meetings and workshops, how might we (HMW) statements help concentrate discussion while still leaving room for a variety of ideas? For example, “How might we condense this 15-step workflow without omitting essential information?” Identify the problem and conduct two exercises to test solutions. For example, three customer surveys conducted over the past six months show a consistent pattern: 30% of customers are dissatisfied with their call center experience, and time-on-call has increased over the same six-month period. Divide the team into two groups. Group 1 writes two new versions of the greeting customer service representatives (CSRs) use when a customer calls. The next step is an A/B test. Group 2 identifies two steps to remove from the current CSR script. The next step is a trial run with CSRs to record time-on-call and customer satisfaction with the call. “Constraint” can be negative, such as a restriction or limitation, but it can also refer to exhibiting control and restraint. By exercising restraint, you and your team can cultivate higher-quality ideas and concentrate on solutions. Rather than generate 50 ideas about how to reconfigure an entire call center setup, it is more productive to focus on two metrics: time-on-task and the customer’s self-rated satisfaction when contacting the call center. By channeling this concentrated energy towards well-defined challenges, your team can then effectively pursue innovative solutions for two closely related issues. Tip 3: Look To Other Domains Other domains or subject areas can be a valuable source of innovative solutions. When facing a challenging design problem, limiting ideas but reaching beyond the immediate domain is a powerful combination. The high-stakes domain of airplane design provides a useful case study of how to simultaneously limit ideas and look to other domains to solve a design problem. Did you know that Otto Lilienthal, a 19th-century design engineer, was the first person to make repeated, successful flights with gliders? Maybe not, but you’ve likely heard of the Wright brothers, whose work launched modern aviation. Why? Lilienthal’s work, while essential, relied on a design based on a bird’s wings, requiring the person flying the glider to move their entire body to change direction. This design ultimately proved fatal when Lilienthal was unable to steer out of a nosedive and crashed. The Wright brothers were bike mechanics who leveraged their knowledge of balance to create a steering mechanism for pilots. By looking outside the “flight domain,” the Wright brothers found a way to balance and steer planes and ultimately transformed aviation. In a similar fashion, Bartholdi, the French artist who sculpted the Statue of Liberty, did not limit himself to looking at statues in Paris. He traveled to Egypt, studied coins and paintings, and drew inspiration from his mother’s face. Designers seeking inspiration should step away from the screen to paint, write a poem, or build a sculpture with popsicle sticks. In other words, paint with oils, not pixels; write with ink, not a keyboard; sculpt with sticks, not white space. On its face, seeking inspiration from other disciplines would seem to contradict Tip 2 above — impose constraints. Examined from another angle, however, imposing constraints and exploring domains are complementary techniques. Rather than list ten random ideas on a whiteboard, it’s more productive to focus on a few solutions and think about these solutions from a variety of angles. For example, recently, I found myself facing a high volume of ideas, source material, and flow charts. While organizing this information was manageable, distilling it into a form others could absorb proved challenging. Rather than generate a list of ten ways to condense this information, I took the dog for a walk and let my eyes wander while strolling through the park. What did I see when my eyes lit upon barren trees? Branches. And what do flow charts do? They branch into different directions. Upon finishing the walk, I jumped back online and began organizing my source material into a series of branched flows. Was this wildly innovative? No. Was this the first time I had drawn flowcharts with branches? Also no. The difference in this case was the application of the branching solution for all of my source material, not only the flow charts. In short, a walk and a nudge from nature’s design helped me escape the constraints imposed by a two-dimensional screen. Stepping away from the screen is, of course, good for our mental and physical health. The occasional light bulb moment is a bonus and one I’m happy to accept. Conclusion Yet these moments alone are not enough. You must channel inspiration by applying practical techniques to move forward with design and analysis lest you become overwhelmed by so many ideas that you become paralyzed and unable to make a decision. To avoid paralysis and reduce the chances of wasting time, I’ve argued against brainstorming, endless lists, and wall-to-wall post-its. Instead, I’ve proposed three practical techniques to boost creativity. Controlled curiosity. From brainstorming to endless scrolling, exposing yourself to high volumes of information is a trap because absorbing information without understanding the purpose or deeper context won’t make you more creative. The solution lies in transforming curiosity into focused exploration. Purposeful curiosity allows you to explore, think, and identify solutions without drowning in a sea of information. Imposing constraints. Brainstorming long lists of ideas might seem creative, but can actually prove more distracting than energizing. The solution is to nurture creativity with structure by limiting the number of ideas under consideration. This structure enhances creativity by focusing idea generation around a few key themes. Look beyond your immediate domain. Otto Lilienthal’s fatal glider crash shows what can happen when solutions are examined through the single lens of one subject area. The solution is to concentrate on innovative solutions for a single issue while reflecting on the problem from various perspectives, such as two-dimensional design, three-dimensional design, or design in nature. Resources How to be More Creative, Hidden Brain Media “Focus Creative Success is Enjoyed through Restricted Choice” by Annie Laure-Sellier and Darren W. Dahl Further Reading on Smashing Magazine “Boosting Up Your Creativity Without Endless Reference Scrolling,” Marina Chernyshova “UX And Design Files Organization Template,” Vitaly Friedman “The Scent Of UX: The Unrealized Potential Of Olfactory Design,” Kristian Mikhel “Fostering An Accessibility Culture,” Daniel Devesa Derksen-Staats

Valve has dismissed widespread reports of a data breach that supposedly compromised the account details of over 89 million Steam users. In a brief but firm post, Valve said it has examined the leaked data and confirmed that Steam’s systems had not been breached, and users did not need to change their passwords or phone numbers. The leak consisted of old text messages containing one-time authentication codes that had all expired. These were linked to the phone numbers they were sent to, but the phone numbers were not linked to any account details. “The leaked data did not associate the phone numbers with a Steam account, password information, payment information or other personal data,” Valve said. “Old text messages cannot be used to breach the security of your Steam account, and whenever a code is used to change your Steam email or password using SMS, you will receive a confirmation via email and/or Steam secure messages. “You do not need to change your passwords or phone numbers as a result of this event. It is a good reminder to treat any account security messages that you have not explicitly requested as suspicious.” Valve said it has not determined the source of the leak, noting that SMS messages like those leaked are unencrypted and pass through multiple providers. Earlier reports had suggested that a vendor used by Valve to send the authentication codes was the source. According to the initial reports, such as this LinkedIn post by Underdark.ai, the data had been posted on the dark web for sale at a price of $5,000. So, we can all rest easy. But it’s a good reminder to turn on two-factor authentication for Steam (and all your online accounts), and to be suspicious of unsolicited messages.

Introduction Hello everyone, I am Akshay Kumar from Kashipur, Uttarakhand, India. I am a 26-year-old CGI artist. I mostly prefer hard-surface 3D modeling and texturing. I have been working in this industry for the last 4 years, and I love doing this work. It's my passion and profession.I used to play many video games and love to watch sci-fi films. I was very curious about how games are made. I was good at art in school, and my elder brother belongs to this industry, which developed a huge interest in 3D. My brother guides me at every stage as he is also a great artist. So I started learning 3D animation from the Gloss Institute of Visual Arts and Animation.I worked on many AR/VR products and PC and VR games at my studio, mostly on texturing. Old Bolt CutterI was looking for gaming props to add to my portfolio, saw an old bolt cutter, and it clicked for me. There were many details in modeling and texturing that were challenging. I was trying to make it look realistic. It was a small prop with a lot of details to work on. It inspired me. I started collecting references of the asset and watched some videos of the inner parts, how they are connected. First, I observed references and started work.ModelingI started working on high poly first. I tried to model every little and big detail in the high poly. My model is 70-75% symmetrical, but for a realistic look, I had to add some imperfections. So I modeled one side and mirrored it. And then added some surface imperfections manually. I did the same for low poly. I tried to keep the model so low-poly that there were no extra edges, and it could cover all details. I kept the mesh on the screws and metal plate to preserve the details. UVsI kept optimizing edge flow to ignore the artifacts in the mesh. I tried to understand the shape to maintain the proper edge loop. So I didn't need to leave edges in the middle. It can be a problem during baking. Good edge flow helped me do a clean and proper unwrapping. I hid the seam, trying to keep no visible seams on the mesh. I set the texel density according to the details using Autodesk Maya and RizomUV to unwarp my model. I divided it into two parts: for big and small shapes.TexturingI divided the prop into two parts to get more details in texturing. I baked the high poly mesh details on the low poly. I used the following settings for baking.I had to work manually on some parts to fix normals and ambient occlusion. Then I started creating base materials. I tried to add details according to the age of the prop: I added some scratches and damages to the surface, AO noise, dirt, and some colour splashes. I imagined how the tool was used and added details accordingly. I focused on the metal of the assets. I searched the metals and understood the nature of metal – how it deals with time, how the real shape is created, where and how it is connected. Is it a single shape or connected with a weld? I worked accordingly.I added rust according to the surface, where dirt would stay on the asset, in which position the asset was kept, where it was in contact with the hand, and if it falls down, where it would hit.For some types of dents, I used some brushes to add depth.I used an anchor point for information from the lower mask, used it for the outer edge of damaged depths, and inverted it to add dirt and rust into the depths.I added some anisotropy to achieve a realistic feel of the material and broke it at some spots.I was also checking the texture maps from time to time to understand if everything was going well or not.Lighting & RenderingAfter completing it from my side, I showed it to some friends – to see if I was missing some details and to get some advice – and worked on that. For better composition, I watched some videos and some artwork on ArtStation – how they showcase their work.I found out which details I have to show in which condition – which angles and intensity of the light will work perfectly. I also played with the value of the camera's field of view and added depth of field. I went with 3-point lighting (key light, fill light, and back light) – and added some manual lights according to the scenes and details I had to showcase.Some details needed reflection, and some needed contrast. I was careful with shadows – they shouldn’t be too hard or soft. I needed to balance it. I tried many scenes, lighting positions, and HDRIs – some worked properly, some didn’t work out.I changed the lights for many renders in Blender's Cycle according to the view. I set up some Compositing nodes, added some glare, adjusted the overall brightness and contrast, and added some shade of vignette.ConclusionThe main challenge was preserving both small and large details within a limited polycount. Every shape should be modeled and connected just like in real life, where dents and scratches go, how to control the intensity of dust and rust, and the level of dirt. Are surface imperfections okay, or do they look odd?I learned a lot from this project. I went through the full game asset creation pipeline – from high poly to low poly. It improved my skills in hard surface modeling – how to optimize a low poly model with better edge flow and no artifacts. I worked more deeply on realistic texturing – understanding smart masks and alpha brushes, finding the right amount of detail (no more, no less), and building a scene and theme that fit the asset. I learned what can be enhanced in compositing and what can be brought out with lighting. I studied how different types of lights behave – which ones need more intensity, which ones need less, the right angles, how they affect the props, and where to place the camera focus.My advice to beginners: break your project into steps. First, you have to figure out what type of artwork you want to work on. You should search for more and more references to gather information. Focus on small details – these are what add realism to your props. Try to understand the shape, details, and life of the asset.Study your references carefully before starting. Gather as many as you can. Set up your workflow and always keep in mind what you want to achieve. Try new things – sometimes they work, sometimes they don’t – but you’ll always learn. These things will bring you new ideas and techniques. Pay attention to small details. At the beginning, we usually avoid the challenging parts – don’t do that. At least try to take them on – it helps you upgrade your skills.Thanks.Akshay Kumar, Hard Surface ArtistInterview conducted by Emma Collins

Mercredi 14 mai, durant quelques heures, Grok, l’agent conversationnel du réseau social X, a fait montre d’une étrange obsession. Dans ses réponses, l’intelligence artificielle (IA) propriété d’Elon Musk a évoqué répétitivement la thèse d’un « génocide blanc » en Afrique du Sud, même lorsqu’on l’interrogeait sur tout autre chose – comme sur le salaire d’un joueur de baseball, ou la méthode pour changer de logiciel. Dans ses réponses, Grok faisait ainsi référence aux meurtres de fermiers blancs, dans ce pays où le taux d’homicides est particulièrement élevé. La thèse selon laquelle la population sud-africaine blanche serait menacée d’un « génocide » est depuis longtemps soutenue par Elon Musk, avant d’être reprise à son compte par Donald Trump. Peu après sa réélection, début février, le président américain s’est ainsi dit favorable à accorder l’asile aux Afrikaners, les descendants des colons européens installés en Afrique du Sud. Les premiers Sud-Africains à répondre à cette invitation sont d’ailleurs arrivés sur le territoire américain mardi 13 mai. Lire aussi | Meurtres de fermiers blancs en Afrique du Sud : le président Cyril Ramaphosa appelle au calme Modifier le fonctionnement des IA Dans ses réponses, l’IA d’Elon Musk ne soutenait pas pour autant l’existence d’un « génocide blanc » : elle présentait au contraire ce concept comme un sujet qui divise, démontant à l’occasion la thèse du génocide, comme l’a constaté le média The Verge. Certains de ces messages ont ensuite été effacés – et, dans la foulée, l’obsession de l’IA pour l’Afrique du Sud corrigée. Beaucoup d’IA génératives sont fermement éduquées à présenter les questions politiques de façon équilibrée. Selon diverses études, Grok aurait ainsi tendance à pencher du côté du centre gauche. Les robots conversationnels sont faits de milliards de neurones artificiels qui ont formé leurs connexions de manière autonome, et dont l’organisation demeure très opaque. Mais il n’est pas impossible de les pousser à privilégier certaines informations : en mai 2024, l’éditeur Anthropic a ainsi réussi à repérer, chez son IA Claude, les neurones portant en eux la mémoire du Golden Gate Bridge de San Francisco. Ses chercheurs sont ensuite parvenus à accentuer le poids de ces neurones au point que l’IA en parlait dans la majorité de ses réponses, y compris quand on l’interrogeait sur d’autres sujets. Lire aussi | Article réservé à nos abonnés La couleur politique des IA dans le viseur de la droite américaine Le Monde Réutiliser ce contenu

Bagging and boosting are two powerful ensemble techniques in machine learning – they are must-knows for data scientists! After reading this article, you are going to have a solid understanding of how bagging and boosting work and when to use them. We’ll cover the following topics, relying heavily on examples to give hands-on illustration of the key concepts: How Ensembling helps create powerful models Bagging: Adding stability to ML models Boosting: Reducing bias in weak learners Bagging vs. Boosting – when to use each and why Creating powerful models with ensembling In Machine Learning, ensembling is a broad term that refers to any technique that creates predictions by combining the predictions from multiple models. If there is more than one model involved in making a prediction, the technique is using ensembling! Ensembling approaches can often improve the performance of a single model. Ensembling can help reduce: Variance by averaging multiple models Bias by iteratively improving on errors Overfitting because using multiple models can increase robustness to spurious relationships Bagging and boosting are both ensemble methods that can perform much better than their single-model counterparts. Let’s get into the details of these now! Bagging: Adding stability to ML models Bagging is a specific ensembling technique that is used to reduce the variance of a predictive model. Here, I’m talking about variance in the machine learning sense – i.e., how much a model varies with changes to the training dataset – not variance in the statistical sense which measures the spread of a distribution. Because bagging helps reduce an ML model’s variance, it will often improve models that are high variance (e.g., decision trees and KNN) but won’t do much good for models that are low variance (e.g., linear regression). Now that we understand when bagging helps (high variance models), let’s get into the details of the inner workings to understand how it helps! The bagging algorithm is iterative in nature – it builds multiple models by repeating the following three steps: Bootstrap a dataset from the original training data Train a model on the bootstrapped dataset Save the trained model The collection of models created in this process is called an ensemble. When it is time to make a prediction, each model in the ensemble makes its own prediction – the final bagged prediction is the average (for regression) or majority vote (for classification) of all of the ensemble’s predictions. Now that we understand how bagging works, let’s take a few minutes to build an intuition for why it works. We’ll borrow a familiar idea from traditional statistics: sampling to estimate a population mean. In statistics, each sample drawn from a distribution is a random variable. Small sample sizes tend to have high variance and may provide poor estimates of the true mean. But as we collect more samples, the average of those samples becomes a much better approximation of the population mean. Similarly, we can think of each of our individual decision trees as a random variable — after all, each tree is trained on a different random sample of the data! By averaging predictions from many trees, bagging reduces variance and produces an ensemble model that better captures the true relationships in the data. Bagging Example We will be using the load_diabetes1 dataset from the scikit-learn Python package to illustrate a simple bagging example. The dataset has 10 input variables – Age, Sex, BMI, Blood Pressure and 6 blood serum levels (S1-S6). And a single output variable that is a measurement of disease progression. The code below pulls in our data and does some very simple cleaning. With our dataset established, let’s start modeling! # pull in and format data from sklearn.datasets import load_diabetes diabetes = load_diabetes(as_frame=True) df = pd.DataFrame(diabetes.data, columns=diabetes.feature_names) df.loc[:, 'target'] = diabetes.target df = df.dropna() For our example, we will use basic decision trees as our base models for bagging. Let’s first verify that our decision trees are indeed high variance. We will do this by training three decision trees on different bootstrapped datasets and observing the variance of the predictions for a test dataset. The graph below shows the predictions of three different decision trees on the same test dataset. Each dotted vertical line is an individual observation from the test dataset. The three dots on each line are the predictions from the three different decision trees. Variance of decision trees on test data points – image by author In the chart above, we see that individual trees can give very different predictions (spread of the three dots on each vertical line) when trained on bootstrapped datasets. This is the variance we have been talking about! Now that we see that our trees aren’t very robust to training samples – let’s average the predictions to see how bagging can help! The chart below shows the average of the three trees. The diagonal line represents perfect predictions. As you can see, with bagging, our points are tighter and more centered around the diagonal. image by author We’ve already seen significant improvement in our model with the average of just three trees. Let’s beef up our bagging algorithm with more trees! Here is the code to bag as many trees as we want: def train_bagging_trees(df, target_col, pred_cols, n_trees): ''' Creates a decision tree bagging model by training multiple decision trees on bootstrapped data. inputs df (pandas DataFrame) : training data with both target and input columns target_col (str) : name of target column pred_cols (list) : list of predictor column names n_trees (int) : number of trees to be trained in the ensemble output: train_trees (list) : list of trained trees ''' train_trees = [] for i in range(n_trees): # bootstrap training data temp_boot = bootstrap(train_df) #train tree temp_tree = plain_vanilla_tree(temp_boot, target_col, pred_cols) # save trained tree in list train_trees.append(temp_tree) return train_trees def bagging_trees_pred(df, train_trees, target_col, pred_cols): ''' Takes a list of bagged trees and creates predictions by averaging the predictions of each individual tree. inputs df (pandas DataFrame) : training data with both target and input columns train_trees (list) : ensemble model - which is a list of trained decision trees target_col (str) : name of target column pred_cols (list) : list of predictor column names output: avg_preds (list) : list of predictions from the ensembled trees ''' x = df[pred_cols] y = df[target_col] preds = [] # make predictions on data with each decision tree for tree in train_trees: temp_pred = tree.predict(x) preds.append(temp_pred) # get average of the trees' predictions sum_preds = [sum(x) for x in zip(*preds)] avg_preds = [x / len(train_trees) for x in sum_preds] return avg_preds The functions above are very simple, the first trains the bagging ensemble model, the second takes the ensemble (simply a list of trained trees) and makes predictions given a dataset. With our code established, let’s run multiple ensemble models and see how our out-of-bag predictions change as we increase the number of trees. Out-of-bag predictions vs. actuals colored by number of bagged trees – image by author Admittedly, this chart looks a little crazy. Don’t get too bogged down with all of the individual data points, the lines dashed tell the main story! Here we have 1 basic decision tree model and 3 bagged decision tree models – with 3, 50 and 150 trees. The color-coded dotted lines mark the upper and lower ranges for each model’s residuals. There are two main takeaways here: (1) as we add more trees, the range of the residuals shrinks and (2) there is diminishing returns to adding more trees – when we go from 1 to 3 trees, we see the range shrink a lot, when we go from 50 to 150 trees, the range tightens just a little. Now that we’ve successfully gone through a full bagging example, we are about ready to move onto boosting! Let’s do a quick overview of what we covered in this section: Bagging reduces variance of ML models by averaging the predictions of multiple individual models Bagging is most helpful with high-variance models The more models we bag, the lower the variance of the ensemble – but there are diminishing returns to the variance reduction benefit Okay, let’s move on to boosting! Boosting: Reducing bias in weak learners With bagging, we create multiple independent models – the independence of the models helps average out the noise of individual models. Boosting is also an ensembling technique; similar to bagging, we will be training multiple models…. But very different from bagging, the models we train will be dependent. Boosting is a modeling technique that trains an initial model and then sequentially trains additional models to improve the predictions of prior models. The primary target of boosting is to reduce bias – though it can also help reduce variance. We’ve established that boosting iteratively improves predictions – let’s go deeper into how. Boosting algorithms can iteratively improve model predictions in two ways: Directly predicting the residuals of the last model and adding them to the prior predictions – think of it as residual corrections Adding more weight to the observations that the prior model predicted poorly Because boosting’s main goal is to reduce bias, it works well with base models that typically have more bias (e.g., shallow decision trees). For our examples, we are going to use shallow decision trees as our base model – we will only cover the residual prediction approach in this article for brevity. Let’s jump into the boosting example! Predicting prior residuals The residuals prediction approach starts off with an initial model (some algorithms provide a constant, others use one iteration of the base model) and we calculate the residuals of that initial prediction. The second model in the ensemble predicts the residuals of the first model. With our residual predictions in-hand, we add the residual predictions to our initial prediction (this gives us residual corrected predictions) and recalculate the updated residuals…. we continue this process until we have created the number of base models we specified. This process is pretty simple, but is a little hard to explain with just words – the flowchart below shows a simple, 4-model boosting algorithm. Flowchart of simple, 4 model boosting algorithm – image by author When boosting, we need to set three main parameters: (1) the number of trees, (2) the tree depth and (3) the learning rate. I’ll spend a little time discussing these inputs now. Number of Trees For boosting, the number of trees means the same thing as in bagging – i.e., the total number of trees that will be trained for the ensemble. But, unlike boosting, we should not err on the side of more trees! The chart below shows the test RMSE against the number of trees for the diabetes dataset. Unlike with bagging, too many trees in boosting leads to overfitting! – image by author This shows that the test RMSE drops quickly with the number of trees up until about 200 trees, then it starts to creep back up. It looks like a classic ‘overfitting’ chart – we reach a point where more trees becomes worse for the model. This is a key difference between bagging and boosting – with bagging, more trees eventually stop helping, with boosting more trees eventually start hurting! With bagging, more trees eventually stops helping, with boosting more trees eventually starts hurting! We now know that too many trees are bad, and too few trees are bad as well. We will use hyperparameter tuning to select the number of trees. Note – hyperparameter tuning is a huge subject and way outside of the scope of this article. I’ll demonstrate a simple grid search with a train and test dataset for our example a little later. Tree Depth This is the maximum depth for each tree in the ensemble. With bagging, trees are often allowed to go as deep they want because we are looking for low bias, high variance models. With boosting however, we use sequential models to address the bias in the base learners – so we aren’t as concerned about generating low-bias trees. How do we decide how the maximum depth? The same technique that we’ll use with the number of trees, hyperparameter tuning. Learning Rate The number of trees and the tree depth are familiar parameters from bagging (although in bagging we often didn’t put a limit on the tree depth) – but this ‘learning rate’ character is a new face! Let’s take a moment to get familiar. The learning rate is a number between 0 and 1 that is multiplied by the current model’s residual predictions before it is added to the overall predictions. Here’s a simple example of the prediction calculations with a learning rate of 0.5. Once we understand the mechanics of how the learning rate works, we will discuss the why the learning rate is important. The learning rate discounts the residual prediction before updating the actual target prediction – image by author So, why would we want to ‘discount’ our residual predictions, wouldn’t that make our predictions worse? Well, yes and no. For a single iteration, it will likely make our predictions worse – but, we are doing multiple iterations. For multiple iterations, the learning rate keeps the model from overreacting to a single tree’s predictions. It will probably make our current predictions worse, but don’t worry, we will go through this process multiple times! Ultimately, the learning rate helps mitigate overfitting in our boosting model by lowering the influence of any single tree in the ensemble. You can think of it as slowly turning the steering wheel to correct your driving rather than jerking it. In practice, the number of trees and the learning rate have an opposite relationship, i.e., as the learning rate goes down, the number of trees goes up. This is intuitive, because if we only allow a small amount of each tree’s residual prediction to be added to the overall prediction, we are going to need a lot more trees before our overall prediction will start looking good. Ultimately, the learning rate helps mitigate overfitting in our boosting model by lowering the influence of any single tree in the ensemble. Alright, now that we’ve covered the main inputs in boosting, let’s get into the Python coding! We need a couple of functions to create our boosting algorithm: Base decision tree function – a simple function to create and train a single decision tree. We will use the same function from the last section called ‘plain_vanilla_tree.’ Boosting training function – this function sequentially trains and updates residuals for as many decision trees as the user specifies. In our code, this function is called ‘boost_resid_correction.’ Boosting prediction function – this function takes a series of boosted models and makes final ensemble predictions. We call this function ‘boost_resid_correction_pred.’ Here are the functions written in Python: # same base tree function as in prior section def plain_vanilla_tree(df_train, target_col, pred_cols, max_depth = 3, weights=[]): X_train = df_train[pred_cols] y_train = df_train[target_col] tree = DecisionTreeRegressor(max_depth = max_depth, random_state=42) if weights: tree.fit(X_train, y_train, sample_weights=weights) else: tree.fit(X_train, y_train) return tree # residual predictions def boost_resid_correction(df_train, target_col, pred_cols, num_models, learning_rate=1, max_depth=3): ''' Creates boosted decision tree ensemble model. Inputs: df_train (pd.DataFrame) : contains training data target_col (str) : name of target column pred_col (list) : target column names num_models (int) : number of models to use in boosting learning_rate (float, def = 1) : discount given to residual predictions takes values between (0, 1] max_depth (int, def = 3) : max depth of each tree model Outputs: boosting_model (dict) : contains everything needed to use model to make predictions - includes list of all trees in the ensemble ''' # create initial predictions model1 = plain_vanilla_tree(df_train, target_col, pred_cols, max_depth = max_depth) initial_preds = model1.predict(df_train[pred_cols]) df_train['resids'] = df_train[target_col] - initial_preds # create multiple models, each predicting the updated residuals models = [] for i in range(num_models): temp_model = plain_vanilla_tree(df_train, 'resids', pred_cols) models.append(temp_model) temp_pred_resids = temp_model.predict(df_train[pred_cols]) df_train['resids'] = df_train['resids'] - (learning_rate*temp_pred_resids) boosting_model = {'initial_model' : model1, 'models' : models, 'learning_rate' : learning_rate, 'pred_cols' : pred_cols} return boosting_model # This function takes the residual boosted model and scores data def boost_resid_correction_predict(df, boosting_models, chart = False): ''' Creates predictions on a dataset given a boosted model. Inputs: df (pd.DataFrame) : data to make predictions boosting_models (dict) : dictionary containing all pertinent boosted model data chart (bool, def = False) : indicates if performance chart should be created Outputs: pred (np.array) : predictions from boosted model rmse (float) : RMSE of predictions ''' # get initial predictions initial_model = boosting_models['initial_model'] pred_cols = boosting_models['pred_cols'] pred = initial_model.predict(df[pred_cols]) # calculate residual predictions from each model and add models = boosting_models['models'] learning_rate = boosting_models['learning_rate'] for model in models: temp_resid_preds = model.predict(df[pred_cols]) pred += learning_rate*temp_resid_preds if chart: plt.scatter(df['target'], pred) plt.show() rmse = np.sqrt(mean_squared_error(df['target'], pred)) return pred, rmse Sweet, let’s make a model on the same diabetes dataset that we used in the bagging section. We’ll do a quick grid search (again, not doing anything fancy with the tuning here) to tune our three parameters and then we’ll train the final model using the boost_resid_correction function. # tune parameters with grid search n_trees = [5,10,30,50,100,125,150,200,250,300] learning_rates = [0.001, 0.01, 0.1, 0.25, 0.50, 0.75, 0.95, 1] max_depths = my_list = list(range(1, 16)) # Create a dictionary to hold test RMSE for each 'square' in grid perf_dict = {} for tree in n_trees: for learning_rate in learning_rates: for max_depth in max_depths: temp_boosted_model = boost_resid_correction(train_df, 'target', pred_cols, tree, learning_rate=learning_rate, max_depth=max_depth) temp_boosted_model['target_col'] = 'target' preds, rmse = boost_resid_correction_predict(test_df, temp_boosted_model) dict_key = '_'.join(str(x) for x in [tree, learning_rate, max_depth]) perf_dict[dict_key] = rmse min_key = min(perf_dict, key=perf_dict.get) print(perf_dict[min_key]) And our winner is — 50 trees, a learning rate of 0.1 and a max depth of 1! Let’s take a look and see how our predictions did. Tuned boosting actuals vs. residuals – image by author While our boosting ensemble model seems to capture the trend reasonably well, we can see off the bat that it isn’t predicting as well as the bagging model. We could probably spend more time tuning – but it could also be the case that the bagging approach fits this specific data better. With that said, we’ve now earned an understanding of bagging and boosting – let’s compare them in the next section! Bagging vs. Boosting – understanding the differences We’ve covered bagging and boosting separately, the table below brings all the information we’ve covered to concisely compare the approaches: image by author Note: In this article, we wrote our own bagging and boosting code for educational purposes. In practice you will just use the excellent code that is available in Python packages or other software. Also, people rarely use ‘pure’ bagging or boosting – it is much more common to use more advanced algorithms that modify the plain vanilla bagging and boosting to improve performance. Wrapping it up Bagging and boosting are powerful and practical ways to improve weak learners like the humble but flexible decision tree. Both approaches use the power of ensembling to address different problems – bagging for variance, boosting for bias. In practice, pre-packaged code is almost always used to train more advanced machine learning models that use the main ideas of bagging and boosting but, expand on them with multiple improvements. I hope that this has been helpful and interesting – happy modeling! Dataset is originally from the National Institute of Diabetes and Digestive and Kidney Diseases and is distributed under the public domain license for use without restriction. The post Strength in Numbers: Ensembling Models with Bagging and Boosting appeared first on Towards Data Science.

Valve has dismissed widespread reports of a data breach that supposedly compromised the account details of over 89 million Steam users. In a brief but firm post, Valve said it has examined the leaked data and confirmed that Steam’s systems had not been breached, and users did not need to change their passwords or phone numbers. The leak consisted of old text messages containing one-time authentication codes that had all expired. These were linked to the phone numbers they were sent to, but the phone numbers were not linked to any account details. “The leaked data did not associate the phone numbers with a Steam account, password information, payment information or other personal data,” Valve said. “Old text messages cannot be used to breach the security of your Steam account, and whenever a code is used to change your Steam email or password using SMS, you will receive a confirmation via email and/or Steam secure messages. “You do not need to change your passwords or phone numbers as a result of this event. It is a good reminder to treat any account security messages that you have not explicitly requested as suspicious.” Valve said it has not determined the source of the leak, noting that SMS messages like those leaked are unencrypted and pass through multiple providers. Earlier reports had suggested that a vendor used by Valve to send the authentication codes was the source. According to the initial reports, such as this LinkedIn post by Underdark.ai, the data had been posted on the dark web for sale at a price of $5,000. So, we can all rest easy. But it’s a good reminder to turn on two-factor authentication for Steam (and all your online accounts), and to be suspicious of unsolicited messages.

It’s in stock now but for how long is impossible to say (Nintendo) Three separate retailers have Nintendo Switch 2 console pre-orders available today, including some new and very expensive game bundles. There have been no big announcements, from Nintendo or anyone else, but after Very had active Switch 2 pre-orders for most of this week other retailers seem to have got stock in too. The big news is that Nintendo themselves are now offering pre-orders for both the console on its own and the Mario Kart World bundle, so there’s obviously been a big delivery recently. There’s no guarantee how long they’ll last but then there’s been very little communication about stock at all during the pre-order process, with no forewarning from Nintendo or anyone else. But with the June 5 launch of the console rapidly approaching it’s a relief to see new stock is arriving all the time. The only official bundle for the Switch 2 is the one with Mario Kart World, which is priced at £429.99. The console alone costs £395.99. As you can see, some retailers have made up their own bundles, which raises the price even further, but at least some of these involve desirable extras, like the camera use for the GameChat feature that Nintendo is keen to promote. Although a lot of pre-orders have gone up this week, the other option is eBay – where scalpers have been posting pre-orders ever since the Switch 2 went on sale. That hasn’t gone down well with fans though, who have repeatedly tried to stop them. Cyberpunk 2077 is a Switch 2 launch title (CD Projekt) Switch 2 pre-orders at Nintendo: Nintendo Switch 2 console – £395.99 Nintendo Switch 2 + Mario Kart World – £429.99 Switch 2 pre-orders at Very: Nintendo Switch 2 console – £395.99 Nintendo Switch 2 + Mario Kart World – £429.99 Nintendo Switch 2 + Mario Kart World + Samsung Micro SD Card – £479 Nintendo Switch 2 + Mario Kart World + Switch 2 Camera – £479 Nintendo Switch 2 + Mario Kart World + Zelda: Tears Of The Kingdom – £494 Switch 2 pre-orders at The Game Collection: Nintendo Switch 2 + Split Fiction + Switch 2 Camera – £484.95 Nintendo Switch 2 + Hogwarts Legacy + Switch 2 Camera – £484.95 Nintendo Switch 2 + Mario Kart World + Split Fiction + Switch 2 Camera – £519.95 Nintendo Switch 2 + Mario Kart World + Hogwarts Legacy + Switch 2 Camera – £519.95 Nintendo Switch 2 + Cyberpunk 2077 + Micro SD Express Card + Switch 2 Camera – £559.95 More Trending Nintendo Switch 2 + Cyberpunk 2077 + Hitman + Micro SD Express Card + Switch 2 Camera – £609.95 Nintendo Switch 2 + Mario Kart World + Cyberpunk 2077 + Hitman + Micro SD Express Card + Switch 2 Camera – £644.95 It’s very hard to predict how long any of this will last, particularly the ones on Nintendo’s websites, as while the Very pre-orders have been around all week there were also ones on John Lewis that lasted mere minutes. If you want to buy from Nintendo bear in mind that you need to have an action Nintendo Switch Online account or you won’t be able to pre-order. At a minimum that costs £6.99 for three months. Mario Kart World is in the official Nintendo bundle (Nintendo) Email gamecentral@metro.co.uk, leave a comment below, follow us on Twitter, and sign-up to our newsletter. To submit Inbox letters and Reader’s Features more easily, without the need to send an email, just use our Submit Stuff page here. For more stories like this, check our Gaming page. GameCentral Sign up for exclusive analysis, latest releases, and bonus community content. This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply. Your information will be used in line with our Privacy Policy

Grand Theft Auto fans have been poring over the second GT 6 trailer and think they've found a particularly controversial feature in Rockstar's upcoming game—the inclusion of childrenTech11:10, 15 May 2025Look, Rockstar, we're almost bored about writing about GTA 6 at this pointAfter a year and a half, Rockstar Games finally released GTA 6's Trailer 2 last week, along with a host of screenshots and character information.We've been watching it regularly to find even more details each time, but some eagle-eyed fans think they've spotted something most of us missed.‌When zooming in on a frame of the trailer, fans think they've found what appears to be a child. That would mark, at least in our recollection, the first time Rockstar has added a youngster to Grand Theft Auto's notoriously adult-centric universe (which may or may not be tied to Red Dead Redemption's).‌Here's all we know.Content cannot be displayed without consentAs you can see in the screenshot posted on X (formerly Twitter) above, it certainly looks as though one character model on the dock, sitting at a table and seemingly flanked by two adults, could be a child.Article continues belowThey're shorter than the two people they're sat with, as well as having a slighter frame, but it seems unlikely we'll see children included in GTA 6 outside of cutscenes.Given how recent the tragedy of the horrific Southport attacks are in the public memory, we imagine this is a snippet from a cutscene and not part of gameplay.There's a good chance Rockstar gets slapped with an 'M' rating by the ESRB in any case, and the game's second trailer showcases sex and violence in equal measure.‌Then there's the likelihood of drugs making an appearance, Rockstar's signature satirical writing and references, and, uh, well the beers Jason and Lucia drink will probably be a red flag, too.GTA 6 now has a release date of May 26 next year, with Rockstar Games announcing a delay a fortnight ago."Grand Theft Auto VI is now set to release on May 26, 2026. We are very sorry that this is later than you expected," the studio said in a statement.Article continues below"The interest and excitement surrounding a new Grand Theft Auto has been truly humbling for our entire team. We want to thank you for your support and your patience as we work to finish the game."For more on the game, we learned how long it's been in development thanks to an ex-developer on the project.For the latest breaking news and stories from across the globe from the Daily Star, sign up for our newsletters.‌‌‌

The European Commission said TikTok might be breaking EU rules on advertisement information provided to users.