Some companies will not want to carry this ethical burden, so they will ignore these kinds of questions. You can see this eschewal of duty in other domains. Cigarette companies know their products cause cancer, but they sell them anyway. Fast food companies know their meals cause obesity and diabetes, but they serve them anyway. When something's very good for business, ethics often take a second seat.

But there will be other companies that accept this ethical burden, and choose to use their powers wisely.

We can call such companies “medicine man companies” — behaving like medicine men for the species.

A medicine man company would observe a given community, society, or even a whole civilization, and try to sense what's ailing it. Then, it would create technological interventions to counteract those ailments. It would use software as a kind of medicine, traveling into the world and subtly altering the behavior of people. A medicine man company would become a new kind of healthcare provider, helping people heal.

In designing interventions to address particular problems, you should understand that you can never simply "fix" a problem. By adding a new element into a system, you increase the complexity of that system, which may have the effect of fixing the problem you saw, but which will also inevitably introduce new and different problems. This is how interventions work. They address one issue, and in doing so, they create new issues, and the world becomes more complex. So if you intervene, do it with humility, knowing that your well-intentioned actions will create unforeseen problems of their own.

So then why act? Why add complexity? If any intervention will create both good and bad, then why intervene at all? Why not simply sit and watch?

We should act because the world is getting crazy, and beautiful interventions are needed.