Toggle light / dark theme

How to use causal influence diagrams to recognize the hidden incentives that shape an AI agent’s behavior.


There is rightfully a lot of concern about the fairness and safety of advanced Machine Learning systems. To attack the root of the problem, researchers can analyze the incentives posed by a learning algorithm using causal influence diagrams (CIDs). Among others, DeepMind Safety Research has written about their research on CIDs, and I have written before about how they can be used to avoid reward tampering. However, while there is some writing on the types of incentives that can be found using CIDs, I haven’t seen a succinct write up of the graphical criteria used to identify such incentives. To fill this gap, this post will summarize the incentive concepts and their corresponding graphical criteria, which were originally defined in the paper Agent Incentives: A Causal Perspective.

A causal influence diagram is a directed acyclic graph where different types of nodes represent different elements of an optimization problem. Decision nodes represent values that an agent can influence, utility nodes represent the optimization objective, and structural nodes (also called change nodes) represent the remaining variables such as the state. The arrows show how the nodes are causally related with dotted arrows indicating the information that an agent uses to make a decision. Below is the CID of a Markov Decision Process, with decision nodes in blue and utility nodes in yellow:

The first model is trying to predict a high school student’s grades in order to evaluate their university application. The model uses the student’s high school and gender as input and outputs the predicted GPA. In the CID below we see that predicted grade is a decision node. As we train our model for accurate predictions, accuracy is the utility node. The remaining, structural nodes show how relevant facts about the world relate to each other. The arrows from gender and high school to predicted grade show that those are inputs to the model. For our example we assume that a student’s gender doesn’t affect their grade and so there is no arrow between them. On the other hand, a student’s high school is assumed to affect their education, which in turn affects their grade, which of course affects accuracy. The example assumes that a student’s race influences the high school they go to. Note that only high school and gender are known to the model.

Please share this event with someone that you care about.

Would you like to make a Donation to Perpetual Life? Your donations help us grow & improve our services. To donate, go to our website http://Perpetual.Life and use the PayPal button at the top right of the page. Thank you for your generous donations, we appreciate it immensely!

“Our task is to make nature, the blind force of nature, into an instrument of universal resuscitation and to become a union of immortal beings.“
- Nikolai F. Fedorov.

We hold faith in the technologies & discoveries of humanity to END AGING and Defeat involuntary Death within our lifetime.

It started with JPL agreeing to land something on Mars – cheaply – and do it in a radically different way. This is how the era NASA called “Faster, Better, Cheaper” began. The documentary film “The Pathfinders” tells the story of a small group of engineers at NASA’s Jet Propulsion Laboratory who did not heed warnings that the audacious challenge of landing on Mars with airbags would likely not be a career-enhancing move.

From relying on a parachute that could not be tested in a way to match the Martian atmosphere to receiving the late addition of an unwanted rover that wouldn’t have looked out of place in a toy store, the Mars Pathfinder mission was a doubter’s dream, taken on by a mostly young group of engineers and scientists guided by a grizzled manager known for his maverick ways.

“The Pathfinders” retraces the journey of this daring mission to Mars that captured the imagination of people around the world with its dramatic landing and its tiny rover – the first wheels ever to roll on Mars.

Documentary length: 60 minutes.

The Universe is a vast place, filled with more galaxies than we’ve ever been able to count, even in just the portion we’ve been able to observe. Some 40 years ago, Carl Sagan taught the world that there were hundreds of billions of stars in the Milky Way alone, and perhaps as many as 100 billion galaxies within the observable Universe. Although he never said it in his famous television series, Cosmos, the phrase “billions and billions” has become synonymous with his name, and also with the number of stars we think of as being inherent to each galaxy, as well as the number of galaxies contained within the visible Universe.

But when it comes to the number of galaxies that are actually out there, we’ve learned a number of important facts that have led us to revise that number upwards, and not just by a little bit. Our most detailed observations of the distant Universe, from the Hubble eXtreme Deep Field, gave us an estimate of 170 billion galaxies. A theoretical calculation from a few years ago — the first to account for galaxies too small, faint, and distant to be seen — put the estimate far higher: at 2 trillion. But even that estimate is too low. There ought to be at least 6 trillion, and perhaps more like 20 trillion, galaxies, if we’re ever able to count them all. Here’s how we got there.

Artificial intelligence; it’s everywhere! Our homes, our cars, our schools and work. So where, if ever, does it stop? And how close to ourselves can our devices reasonably get? For this video, Unveiled uncovers plans to use human brain implants to improve the performance of our brains! What do you think? Are neural implants a good thing, or a bad thing?

This is Unveiled, giving you incredible answers to extraordinary questions!

Find more amazing videos for your curiosity here:
What If Humanity Was A Type III Civilisation? — https://www.youtube.com/watch?v=jcx_nKWZ4Uw.
Why the Microverse Might Be a Reality — https://www.youtube.com/watch?v=BF6n-bjYr7Y

Are you constantly curious? Then subscribe for more from Unveiled ► https://goo.gl/GmtyPv.

Echolocation is a skill we usually associate with animals such as bats and whales, but some blind humans also use the echoes of their own sounds to detect obstacles and their outlines. Some use the tapping of a cane or the snapping of their fingers to make the necessary noise, while others use their mouths to make a clicking sound.

Despite how useful this skill can be, very few blind people are currently taught how to do it. Expert echolocators have been trying to spread t… See more.


With enough training, most humans can learn how to echolocate, using their tongue to make clicking sounds and interpreting the echoes that come back, reflected from the surrounding environment.

In as few as 10 weeks, researchers have been able to teach participants how to navigate obstacles and recognize the size and orientation of objects using the rebounding calls of their clicks.

After some serious number crunching, a UBC researcher has come up with a mathematical model for a viable time machine.

Ben Tippett, a mathematics and physics instructor at UBC’s Okanagan campus, recently published a study about the feasibility of . Tippett, whose field of expertise is Einstein’s theory of general relativity, studies black holes and science fiction when he’s not teaching. Using math and physics, he has created a formula that describes a method for time travel.

“People think of time travel as something as fiction,” says Tippett. “And we tend to think it’s not possible because we don’t actually do it. But, mathematically, it is possible.”

Mike LorreyThe arguments I put into my article in The Space Review for the Space Force are valid to this discussion. https://www.thespacereview.com/article/3576/1


Real-world examples make the abstract description of machine learning become concrete.

In this post you will go on a tour of real world machine learning problems. You will see how machine learning can actually be used in fields like education, science, technology and medicine.

Each machine learning problem listed also includes a link to the publicly available dataset. This means that if a particular concrete machine learning problem interest you, you can download the dataset and start practicing immediately.

Understanding the mind and how thinking occurs has been a challenge for philosophers, scientists, theorists, educators, and artists throughout history. Until recently, ideas about how we learn have been mainly theoretical and intuitive. However, with ongoing advances in neuroscience, considerable progress is occurring. As a result, a paradigm shift is taking hold in human cognition, pointing to a new science-based understanding about the way we think and, ultimately, the way we learn.

This paradigm shift — a move away from traditional notions of the mind to an “embodied cognition” model of human thinking and learning — is the subject of a new book “Movement Matters: How Embodied Cognition Informs Teaching and Learning”. The book is summarised as follows:

“Experts translate the latest findings on embodied cognition from neuroscience, psychology, and cognitive science to inform teaching and learning pedagogy.”

This London Futurists webinar featured the co-editors of this book, Sheila Macrine, Professor of STEM Education & Teacher Development at the University of Massachusetts, Dartmouth, and Jennifer Fugate, Associate Professor in the Department of Health Service Psychology, at Kansas City University.