Executive summary

Initial sense-making in small groups (e.g. core teams) is currently the most underserved need in online governance. It’s currently addressed with solutions like interviews, workshops and surveys, none of which enable fast and scalable sense-making. This problem space relies heavily on facilitation skills of people responsible for governance or strategy. Instead of optimising voting and delegation systems (that re-create the problems of the outdated real-life democracies), the most impactful solutions will address the problem of facilitation. Modern LLMs have the potential to make it much easier for organisations to identify tensions, goals, and generate collective intelligence artefacts that would ensure higher alignment and legitimacy of their decisions. This paper is an attempt to bridge the gap between decentralised governance and deliberative democracy as they face a lot of common challenges and both can benefit from deploying AI / LLMs at scale.

Introduction

Decentralized organizations aim to enable participatory and distributed governance and decision-making. However, facilitating effective collective sense-making in this context remains a significant challenge. Sense-making is the collaborative process of developing shared understanding and framing of complex issues to inform decisions and actions. It is a core capability of an organization or society.

Inadequate sense-making can lead to an increasing gap between the complexity of the environment and the ability to understand and address it. This can result in apathy, resignation, or oversimplification of problems. Sense-making occurs at both personal and collective levels. Personal sense-making requires cognitive capacity, time, attention, and open-mindedness. Collective sense-making involves aggregating preferences and consolidating expertise to make competent group decisions, resulting in shared worldviews and lore.

In decentralized organizations, sense-making is critical for:

• Formulating shared purpose, priorities, and goals

• Deliberating on key decisions and tradeoffs

• Interpreting ambiguous information and dynamic environments

• Adapting strategies based on feedback and learning

However, current approaches to collective sense-making in decentralized groups face limitations. Engaging diverse members, eliciting quality inputs, and synthesizing insights is time- and effort-intensive. Lack of structure and facilitation skills can lead to inefficient and inconclusive processes, hindering groups from leveraging their collective intelligence and resulting in misalignment and lack of clarity.

Literature review

Decision-making

Aviv Ovadya suggests that to address urgent global crises and ensure that decision-making systems incorporate human values, we must improve these systems to be competent, aligned, and robust — in other words, wise. A wise decision-system should be skilled at identifying and evaluating options, aligning across conflicting values, and remaining resilient in the face of real-world complexity and adversarial forces. Improving decision-systems requires integrating insights from various domains, considering the properties of the decision-system itself, and the legitimacy and trust it holds in the world outside the system.

Facilitation

Two papers on facilitation in the context of deliberative democracy are particularly useful: (1) Facilitators: The Micropolitics of Public Participation and Deliberation (Escobar, 2019) and (2) Diversity in Facilitation: Mapping Differences in Deliberative Designs (von Schneidemesser et al., 2023). Both papers highlight the importance of facilitators in shaping participatory and deliberative processes. Von Schneidemesser et al. focus on comparing different facilitation approaches and their impact on deliberative outcomes, while Escobar takes a broader view of the role of facilitators in democratic practices and the micropolitics of their work. The papers converge on the idea that facilitation is not a one-size-fits-all practice and that different approaches may be suitable for different contexts and desired outcomes. However, Escobar's chapter goes beyond the comparative approach by delving into the political nature of facilitation work and the challenges of institutionalising and professionalising this role within governance structures. Escobar's focus on the micropolitics of facilitation provides a valuable lens for understanding the complex power dynamics at play in participatory processes.

Based on the insights from the academic articles, here are some recommendations for practitioners and developers building innovative facilitation solutions:

1. Tailor the facilitation approach to the context and desired outcomes. Different facilitation methods have their strengths and weaknesses. Practitioners should carefully consider the goals, group size, and participant characteristics when selecting a facilitation approach. Developers should design flexible solutions that can accommodate various facilitation styles and techniques.

2. Prioritise psychological safety and trust-building. It's important to create a safe space where participants feel heard and respected. Facilitators should focus on building trust and navigating power dynamics. Developers should incorporate features that promote inclusivity, anonymity (when appropriate), and clear communication guidelines.

3. Embrace creative tension and divergent thinking. While synthesising information and finding common ground is important, breakthrough ideas often come from outliers. Facilitators should encourage participants to express diverse perspectives and allow for creative tension. Developers can design tools that highlight outliers and help groups explore ideas from multiple angles.

4. Augment human facilitators with AI, but don't replace them. The articles discuss the potential of AI and LLMs to support facilitation, while the experts caution against over-relying on technology. Practitioners should view AI as a tool to assist with collecting inputs and synthesis, but not as a substitute for human judgement and empathy. Developers should focus on creating AI-powered solutions that complement human facilitators and enhance their effectiveness.

5. Design for intervention, not just participation. The articles challenge the notion that participation should be the primary metric of success. Instead, practitioners should design governance systems that allow for timely intervention when things go off track. Developers can create tools that provide real-time insights and early warning signals to help groups course-correct as needed.

6. Balance efficiency with robustness and resilience. While AI can help gather input quickly and generate ideas, the experts warn against optimising solely for speed and efficiency. Practitioners should ensure that there is sufficient time for deliberation and relationship-building, as this can lead to more robust and resilient outcomes. Developers should create solutions that encourage deeper engagement and help groups navigate complexity.

By following these recommendations, practitioners and developers can create more effective and context-appropriate facilitation solutions that leverage the strengths of both human facilitators and AI-powered tools. The goal should be to empower groups to make better decisions, navigate complexity, and achieve their desired outcomes while prioritising trust, inclusivity, and resilience.

DAO governance

In his seminal post from 2022, Vitalik Buterin wrote about making better decisions in concave environments, where pluralism and even naïve forms of compromise are on average likely to outperform the kinds of coherency and focus that come from centralization. He suggests two things that can help ensure that an org will be meaningfully decentralized:

1. A truly high level of autonomy for units, where the units accept resources from the core and are occasionally checked for alignment and competence if they want to keep getting those resources, but otherwise act entirely on their own and don't "take orders" from the core.

2. Highly decentralized and diverse core governance. This does not require a "governance token", but it does require broader and more diverse participation in the core. Normally, broad and diverse participation is a large tax on efficiency. But if pods are highly autonomous and the core needs to make fewer decisions, the effects of top-level governance being less efficient become smaller.

Buterin further explains how this fits into the "convex vs concave" framework: the (more decentralized) top level is concave, while the (more centralized within each pod) bottom level is convex.

As Lavande from Optimism suggested in their ETH Denver presentation, DAOs need to create prioritisation frameworks: "Strategy is hard to define collectively. DAOs without a strategy tend to be directionless as attention and spend become increasingly divided. Focused DAOs usually set strategy via prominent leadership with a high degree of influence. Influence is effective at the beginning of a DAO's life, but as the initial leaders phase themselves out, the succession problem arises. Prioritization frameworks teach the community how to make strategic decisions themselves. These can be high level (values, guiding principles, or scope) or granular (OKRs / KPIs). Only 55% of DAOs analyzed had a formal mission statement, vision, values, or public roadmap. KPIs were only implemented at 40% of the DAOs analyzed. Prioritization frameworks are an important tool to empower the community to make decisions on their own, increasing the resilience of the DAO over time."

Lavande also emphasized the need for DAOs to develop mechanisms to collect and incorporate contributor feedback, stating that there is usually plenty of top-down communication (core team to community) but limited communication from community to core team.

Lavande highlighted the importance of legitimacy for DAOs: "The difference between crisis and collapse is legitimacy. The presence of conflict (either external or internal) does not appear to be a distinguishing factor in a DAO's success or failure. My hypothesis is that legitimacy was the key difference. It can be established by brute force, continuity, fairness, process, performance, or participation. Since DAOs are closer to sovereigns than corporations, their legitimacy is created endogenously. A loss of legitimacy can result in a complete collapse of the DAO, since there is no external mechanism to reinforce it. DAOs should do everything possible to avoid negating any of the sources of their legitimacy."

The Governance Geeks Gathering, organized by Radicle at ETH Istanbul in late 2023, identified a few very similar challenges:

1. Need for clear shared purpose: Many of the challenges or groups of challenges stemmed from the lack of a clear shared purpose/vision for the DAO. Not having a clear shared purpose can lead to:

   • Inability to design effective governance & incentive mechanisms

   • Lower transparency on roles, responsibilities and strategy alignment

   • Difficulties finding "DAO/Product fit"

   • Poor process for off-loading work to the community

2. Capturing community insights: There is a need for better community/stakeholder analysis tools and services (e.g., ways to get better insights into how motives/incentives differ between different stakeholders). Better community insights = more evidence to signal how we should be designing governance mechanisms.

The Governance Geeks Gathering also emphasized the importance of a clearly defined purpose for DAOs: "Many projects strive for a purpose- or mission-aligned governance system - but the understanding of a shared "purpose" is often not clear to everyone within an org. A clearly defined purpose acts as a north-star for governance coordination, development strategy and sustainability goals in a DAO. Having a clearly defined purpose allows different teams better coordinate their own strategies and initiatives to align with the shared DAO purpose. It also helps sub-DAOs better understand their relationship with the DAO, as well as expectations regarding funding and broader support in the short- & long-term. Not having a clear purpose to rally around can cause confusion and chaos around decision-making in good and bad times. The longer a DAO goes without having a clearly defined purpose, the more it risks introducing new systems that can introduce constraints or other issues with governance in the long run."

However, the Gathering also acknowledged the difficulty in curating a shared purpose: "Curating a shared purpose can be difficult given the complexity of DAO structure and roles. Identifying who should lead this work or what stakeholders should be involved in its creation is tricky and will likely look different for each DAO. Founders often don't want to take on this responsibility, although they have the best context. Trying to include the broader community in this process is important but strenuous given the sheer coordination effort needed to collect input from different stakeholders and boil them down into a single paragraph or sentence."

Understanding the problem

Over the last two months, I interviewed around 15 practitioners, including governance leads, facilitators, and consultants, and created a map of the decision-making process. I found that

The decision-making process can be described as four distinct stages that involve specific stakeholders and jobs to be done. In most DAOs the initial sense-making relies too much on manual facilitation, making it difficult to capture the insights of members with the highest context, leading to lower alignment of the whole DAO.

Key struggles during the initial sense-making stage include:

• Getting the right people involved (with high context)

• Knowledge management as a key limitation

• Lack of skills and/or experience in facilitation

• Conflict avoidance

• Some people are not used to Miro

Key struggles during the validation stage include:

• Reaching out to delegates (doxxed only in big DAOs with >5000 members)

• Wisdom vs. popularity (people don't share the hard truth)

• Low engagement (only 10% fill out surveys)

• Processing of input takes a lot of time

• Fragmented communities are hard to engage and align

• Need to assign different weights

Insights from specific cases

FTW DAO emphasizes collaborative decision-making within its council, aiming for inclusivity through tools like Tally for surveys. However, they face challenges in encouraging independent thinking among council members. This case underscores the classic tension between broad participation and decisive action.

Key takeaways:

• Collaborative decision-making is important for inclusivity

• Tools like Tally can facilitate surveys and gather input

• Encouraging independent thinking among council members can be challenging

• There is a tension between broad participation and decisive action

DAOstar employs a seasonal planning process that integrates forum discussions, roundtable discussions, and Typeform surveys to craft actionable seasonal goals aligned with the collective vision of its members. This approach highlights the importance of continuous engagement and feedback in shaping the DAO's trajectory.

Key takeaways:

• Seasonal planning process helps align goals with the collective vision

• Integrating various engagement methods (forum discussions, roundtable discussions, surveys) is crucial

• Continuous engagement and feedback are essential for shaping the DAO's trajectory

Metagov's rebranding initiative showcases the role of community engagement and feedback in reshaping an organization's identity. Their board made the ultimate decisions, but they utilized surveys and workshops to gather feedback, refine ideas in Google Docs, and maintain alignment with the organization's mission, vision, and values. Metagov's rebranding exercise involved multiple stages of community feedback, synthesis, and refinement, ultimately leading to a reassessment of the organization's mission and values.

Key takeaways:

• Community engagement and feedback are crucial in reshaping an organization's identity

• Surveys and workshops can be used to gather feedback and refine ideas

• Multiple stages of feedback, synthesis, and refinement lead to better alignment with the organization's mission and values

• Rebranding exercises can lead to a reassessment of the organization's mission and values

Gitcoin leverages their governance forum on Discourse and introduces a Citizen Grants Program to enhance governance and community engagement. The challenge of making sense of Gitcoin's complex ecosystem is met with innovative solutions like interactive delegate meetings and the use of collaborative tools for synthesis and communication. They offer resources to enhance member understanding of complex topics and encourage interactive delegate meetings to discuss key priorities. CoachJ's efforts at Gitcoin revolve around distilling complex ecosystem updates and aligning diverse stakeholder interests through targeted information dissemination.

Key takeaways:

• Discourse forum is the main space for Gitcoin's governance and community engagement

• Interactive delegate meetings and online collaboration tools are used to make sense of the fragmented ecosystem

• Offering resources to enhance member understanding of complex topics is important

• Distilling complex updates and aligning diverse stakeholder interests through targeted information dissemination is not easy

Thank Arb and Ethelo piloted a Community Engagement and Evaluation System across the Arbitrum ecosystem, but encountered several issues:

1. Data quality: The pilot was susceptible to sybil attacks, highlighting the need for more stringent and ongoing data validation protocols in web3 contexts.

2. Engagement overload: The pilot's iterative campaign structure led to information overload and survey fatigue, limiting the diversity of responses.

3. User experience: Ethelo's older front-end and user interface may not be optimally suited for web3 users, requiring the development of more intuitive and web3-fitting user experiences.

4. Participant segmentation: Directing respondents to Ethelo through the Thank Arb system made it difficult to separate responses between DAO Delegates and DAO Community Members.

Key takeaways:

• Web3 ecosystems require robust data validation and quality checks to mitigate sybil attacks

• Targeted engagement and concise information presentation are crucial to avoid survey fatigue

• User-centric design and seamless platform access are essential for effective participation

• Separating delegate and community member responses requires careful entry conditions and segmentation

The Thank Arb x Ethelo case study underscores the challenges of adapting traditional engagement systems to web3 contexts, emphasizing the need for tailored solutions that address data integrity, user experience, and participant segmentation issues.

These case studies provide valuable insights into the challenges and strategies employed by various DAOs in their decision-making and sense-making processes. They highlight the importance of community engagement, continuous feedback, collaborative tools, and targeted information dissemination in navigating the complexities of decentralized governance.

Facilitators play an important role in ensuring internal inclusion and navigating complex power dynamics. A few interviews with experienced facilitators Andrew Gray (who organised many citizen assemblies in the UK), Andy Paice, as well as Camille Canon (the co-founder of Apiary) provided key insights and perspectives on facilitation. Facilitation is crucial for enabling collective work through inclusive, meaningful, and productive conversations. It is about creating a safe space, building trust, and allowing for creative tension. Participation should not be the primary metric of success for governance; systems should allow for intervention when things go off track.

Key facilitation challenges:

1. Time constraints: Even with lengthy processes like citizens assemblies (e.g., 30 hours), facilitators often feel there isn't enough time to delve deeply into issues and generate the best recommendations.

2. Balancing individual and group needs: Facilitators must create a safe space for individuals to share their thoughts while also guiding the group towards a common understanding and solution.

Facilitation tools and methods:

1. Low-tech tools: Sending out surveys on paper, using flip charts, and sticky notes for in-person facilitation to ensure inclusivity for all participants.

2. Digital tools: Mentimeter and Slido help to gather questions and input from participants in real-time.

3. Polis: A tool for mapping opinions and understanding how people think about an issue. While it can be useful for facilitators, some participants find it boring or difficult to understand. It is just one input among many in a citizens assembly.

4. AI and LLMs have limitations in facilitation, as they may not replicate the human element of sensing emotions and navigating group dynamics. AI-assisted facilitation can save time on preceding stages of decision-making but should not replace proper deliberation and human facilitators.

Camille Canon highlighted the importance of allowing individuals to feel heard and the potential issues with AI-powered decision-making moving groups further away from their true thoughts and frustrations:

"Facilitation serves two purposes: information sharing and allowing individuals to feel heard/psychological relief. The latter may be difficult to replicate with a chatbot."

"It's kind of counterintuitive, but it's often the outliers of the group that is actually the point of most interest and something that moves the group forward with a decision. ... With multiple layers of decision-making that play into the middle, we ironically get further and further away from the actual truth of the group's thoughts and frustrations."

Seeing the market

I created a Wardley Map of sense-making tools to visualize the landscape using two dimensions: the place in the supply chain (frontend, admin/research-focused tools, and infrastructure) and the maturity of innovation. The sources of innovation are identified by colour.

Commodity tools like Zoom, Miro, and Discourse have limitations, as they often lead to passive participation, with only a few people actively expressing their opinions while others simply listen or read.

In the DAO tooling space, there is a noticeable bias towards voting mechanisms, which represent the final stage of decision-making. This focus on voting may stem from the fact that it presents a more well-defined problem for engineers compared to the complexity of facilitation. As a result, sense-making in DAOs is often done manually and is difficult to scale.

Many DAOs seem to believe that they need "on-chain tools" and overlook the existing knowledge, frameworks, and tools developed for sense-making over the years. In particular, there are numerous tools designed for citizen engagement and deliberative democracy that could be beneficial for DAOs: Your Priorities, Ethelo, All Our Ideas, Consider.it, etc. Additionally, frameworks like Sociocracy and Cynefin could really help DAOs step up their sense-making process.

The mapping of this space reveals that DAOs might benefit from looking beyond their immediate ecosystem and embrace proven tools and frameworks from other domains to enhance their sense-making and deliberation. By doing so, they can address the limitations of manual facilitation and scale their collective intelligence more effectively.

Daniel Ospina

@_Daniel_Ospina

Tools that use AI to automate voting are likely going to get mass adoption, which is really dangerous as they don't represent better decisions. Instead we end up biased on generic western preferences and limited past behaviour personal data.

Deliberation and sortition is where…

23

3:37 PM • Mar 19, 2024

What success will look like

If the problem of slow and unscalable collective sense-making in DAOs is solved, the governance experience (GX) would change dramatically. Instead of focusing on optimizing voting and delegation systems, which may recreate the problems of real-life democracies, we should build governance tools that address the problems of collective sense-making that happens before the proposal by making facilitation at scale much easier, which is impossible without leveraging ML/AI.

Such tools will expand the online collaboration stack, complementing existing use cases with generative collective intelligence.

AI-powered Collective Dialogue Systems, or (more broadly) Group Decision Support Systems have the potential to make it easier for organizations to:

• Identify tensions and organisational drivers (inspired by S3)

• Create prioritization frameworks or scaffolding for decision-making

• Help to identify common ground, surface insights, self-reflect on group experiences, and map perspectives.

Just as advances in machine learning have led to new ways of interacting with computers by shifting from actions to intentions, advances in collective dialogue and group decision support systems can help embed new forms of governance and agency at various levels of society:

1. Governance & Conflict: Deploying these systems to meaningfully include the broader public in decision-making processes.

2. 'Corporate democracy': Supporting a new form of corporate governance that goes beyond shareholders and elite stakeholders to include users, employees, and the impacted public.

3. Media & Understanding: Enabling collective introspection, helping a public 'know itself', identify common ground, and better navigate internal and external challenges.

The success of solving the collective sense-making problem in DAOs will be marked by the emergence of more efficient, inclusive, and impactful decision-making processes that leverage AI-powered solutions to foster collaboration, understanding, and agency at various levels of society.

Recommendations

Based on my research, here are a few recommendations for Developing AI-powered Collective Sense-Making Systems:

1. Establish clear ownership and leadership within the project to drive the development and implementation of these systems, avoiding delays due to a "tragedy of the commons" situation.

2. Engage the community in the creation and ratification of the system's purpose to ensure legitimacy. This can be achieved through surveys, stakeholder interviews, and off-chain consensus voting.

3. Consider engaging a third-party facilitator to assist in the research, drafting, and ratification process, as they can help elicit more candid and honest feedback from stakeholders.

4. Recognize that decision-making is fundamentally driven by the will and goals of the people involved. AI should be used to assist in identifying the collective will and fostering alignment, rather than autonomously setting goals.

5. Focus on using AI to define key results and manage operations, while ensuring that overarching goals remain human-driven. Strong alignment around goals may reduce the need for frequent voting.

6. Explore alternative approaches to representative voting, such as distributing decision-making authority based on roles and expertise (e.g., using the Hats Protocol).

7. Develop standardized Process Cards and Run Reports to facilitate learning, interoperability, and evaluation among process innovators, executors, funders, adopters, and researchers.

8. Investigate the potential for institutionalizing successful processes into resilient networks, perhaps documented through standard 'Structure Sheets' that designate how processes defined in process cards interact.

9. Invest in research to create evaluation protocols and metrics for AI-augmented collective sense-making systems, understand the impacts of different design decisions, and develop increasingly better systems for various purposes and contexts.

10. Ensure consistent messaging and narrative distribution regarding the DAO's purpose, both within and outside the organization, to maintain clarity and alignment.

By following these recommendations, DAOs and other organizations can develop effective AI-augmented collective sense-making systems that foster alignment, legitimacy, and continuous improvement while maintaining a human-centric approach to decision-making.