How intermittent breaks in interaction improve collective intelligence | PNAS

Many human endeavors—from teams and organizations to crowds and democracies—rely on solving problems collectively. Prior research has shown that when people interact and influence each other while solving complex problems, the average problem-solving performance of the group increases, but the best solution of the group actually decreases in quality. We find that when such influence is intermittent it improves the average while maintaining a high maximum performance. We also show that storing solutions for quick recall is similar to constant social influence. Instead of supporting more transparency, the results imply that technologies and organizations should be redesigned to intermittently isolate people from each other’s work for best collective performance in solving complex problems.

People influence each other when they interact to solve problems. Such social influence introduces both benefits (higher average solution quality due to exploitation of existing answers through social learning) and costs (lower maximum solution quality due to a reduction in individual exploration for novel answers) relative to independent problem solving. In contrast to prior work, which has focused on how the presence and network structure of social influence affect performance, here we investigate the effects of time. We show that when social influence is intermittent it provides the benefits of constant social influence without the costs. Human subjects solved the canonical traveling salesperson problem in groups of three, randomized into treatments with constant social influence, intermittent social influence, or no social influence. Groups in the intermittent social-influence treatment found the optimum solution frequently (like groups without influence) but had a high mean performance (like groups with constant influence); they learned from each other, while maintaining a high level of exploration. Solutions improved most on rounds with social influence after a period of separation. We also show that storing subjects’ best solutions so that they could be reloaded and possibly modified in subsequent rounds—a ubiquitous feature of personal productivity software—is similar to constant social influence: It increases mean performance but decreases exploration.

Collective intelligence—the ability of collectives of individuals to solve problems well—has emerged as an important interdisciplinary area of study with applications in understanding and supporting the performance of groups and teams (1), networks (2), crowds (3⇓⇓–6), financial markets (7), prediction markets (8), innovation contests (9), and democracies (10, 11), as well as collectives of nonhuman organisms (e.g., ref. 12). Across these diverse and important settings, a fundamental question is this: How does social influence—exposure of solvers to each other’s behavior or solutions through interacting—affect collective intelligence?

In this work, we conduct randomized experiments to study how collective intelligence is affected by two frequently experienced impacts of technology use: changes to the temporal nature of social influence (from intermittent social influence, which is more characteristic of face-to-face communication, to constant social influence, characteristic of “always-on,” transparency-enhancing communication technologies) and storage and quick recall of a solver’s current best solution to a problem (which in effect increases the influence of a solver’s past solutions on their current solution).

Past research shows that social influence leads individuals to adopt their peers’ opinions and copy their solutions to problems (5), especially under conditions of network clustering (13, 14), leading to a loss of aggregate diversity. Under certain conditions—performing simple estimation tasks or searching simple solution spaces with clear performance feedback—more efficiently connected networks of problem solvers can collectively outperform disconnected individuals (12, 15, 16) and inefficient networks (17, 18), as people learn from each other and better solutions spread rapidly. In general, however, and especially for complex and uncertain problems, maintaining and integrating diverse information and perspectives is a critical driver of collective performance (1, 9, 19). Social influence and in particular network clustering can result in too much local copying behavior, driving out beneficial diversity and resulting in a collective convergence on a suboptimal solution (2, 5, 7, 17, 18). For example, sharing ideas in the early stages of a brainstorming task has been shown to reduce the number and quality of ideas produced (20).

Working together in clusters or groups does offer other benefits for handling large problems, despite the tendency for connected individuals to underexplore solution spaces. In particular, groups are capable of handling complex problems that individuals themselves cannot (21, 22). Innovation and invention are also widely thought to be social processes, in which ideas or partial ideas from multiple individuals are recombined (23, 24).

A major unresolved question in collective intelligence in complex tasks is thus whether it is possible to get the benefits of social influence and network clustering (collective learning) without the associated costs (premature convergence on a suboptimal solution). Here, we report on an experimental study that provides evidence that it is indeed possible, and moreover that conditions typical of real (as opposed to laboratory) face-to-face social networks result in both benefits.

We study the performance of sets of three individuals (hereafter “triads”) completing the Euclidean traveling salesperson problem (TSP), which involves finding the shortest path among symbols representing cities on a synthetic 2D map presented visually. The TSP is NP-hard (nondeterministic polynomial time hard) (25) and characterized by many local optima (26); thus, like other tasks thought to be good models for complex problems (27), solution spaces for the TSP are “rugged” in that simple hill climbing will generally fail to produce a good solution. Although feasible for human subjects (28), finding the globally optimal solution is not trivial and is expected to benefit from more—or more efficient—collective exploration. In our study, each TSP map included 25 different cities; a full path included 25 “legs” of the journey, each connecting a pair of cities. In a single trial, our subjects completed the task 17 times (“rounds”) in a row and thus were able to refine their solution and, depending on the experimental treatment they were assigned to, learn from the other members of their triad.

Our experimental treatments were inspired by the fact that outside of the laboratory real face-to-face communication ties are not constant: Even strong social ties involve intermittent interaction punctuated by time apart (29). Thus, we conducted a three-way randomization with respect to how much network ties within the triad are “on.” One-third of our subjects were assigned to a constant ties (CT) condition, in which they could see the solutions of their neighbors every round of the trial. One-third of our subjects were assigned to an intermittent ties (IT) condition, in which they were able to see their neighbors’ solutions every three rounds (on rounds 4, 7, 10, 13, and 16). The final third were assigned to a no ties (NT) condition, in which subjects could never see their neighbors’ solutions.

In wisdom of the crowd-type tasks (with applications in estimation and prediction), scholars focus on the mean (or other measure of central tendency) of a collective of estimates (3, 5, 8). In complex problem-solving settings such as ours, in addition to the quality of the mean solution, the quality of the best solution produced in a collective is often of critical importance (with applications in, e.g., brainstorming, crowdsourcing, and innovation) (9, 30). In this latter context, scholars have been particularly focused on whether a collective finds the global optimum to a complex problem (2, 17). We therefore consider both performance metrics—best solution and mean solution—in our study.

Because NT triads lack social influence among solvers, prior literature predicts that NT triads would generate more diverse solutions and thus find the optimum solution in more trials than CT triads (9, 17), but at the expense of having an inferior mean solution to that of CT triads (16, 17). Our findings bore out these predictions. Strikingly, as we discuss below, we found that IT triads showed the positive features of both CT and NT triads: They found the optimum solution as frequently as NT triads, but with a higher quality mean solution like CT triads.

CT triads found the optimal solution in 33.3% of trials, IT triads found the optimum in 48.3% of trials, and NT triads found the optimum in 44.1% of trials (the difference between IT and CT was significant after controlling for covariates in a logistic regression; see Table 1 for full models and Materials and Methods for more details). Whether or not a group found the optimum, the best solution found in IT triads and NT triads was significantly better (shorter) than the best solution found in CT trials [log(1+difference from optimal distance) in CT was worse than IT by 0.285, P < 0.001 and NT by 0.211, P < 0.001]; IT and NT triads were not statistically different.

Although social influence reduces exploration and thus depresses the quality of top solutions, it is expected to improve the quality of the mean solution by allowing players with very poor solutions to adopt better solutions from their neighbors (15, 16). We find that to be the case (see Table 1, column 4). The mean solution (all solutions from all triad members across all 17 rounds of a trial) in IT triads was as good as the mean solution in CT triads. The mean solution in NT triads was worse than in IT triads [log(1+difference from optimal distance) was 0.351 longer, P < 0.001] and CT triads (0.449 longer, P < 0.001).

As expected, more social influence resulted in less diversity of solutions. The mean number of unique solutions found by a triad over all 17 rounds was highest in NT triads (30.5), followed by IT triads (27.5) and CT triads (21.4). However, the greater diversity of NT triads did not result in greater performance. Although NT triads found 1.108 times more unique solutions than IT triads (Poisson, P = 0.010), they did not find the optimum more frequently (indeed, NT triads found the optimum less frequently than IT triads, but the difference was not statistically significant).

IT triads displayed a balance between learning from peers (through social influence) and trying diverse new solutions (through independent exploration). In IT triads, answers within a triad alternately became more similar to each other (on rounds in which they could see each other’s answers) and became more different from each other (on rounds in which they could not see each other’s answers), exploring from new starting points (Fig. 1). This contrasts with both other treatments in which the answers within a triad largely became more similar to each other over time on average. In NT triads, answers’ becoming more similar to each other reflects only independent convergence on similar answers, while in CT triads becoming more similar to each other is also the result of social influence.

As pure strategies at the individual solver level, both independent exploration and social influence can lead to “getting stuck” at a suboptimal solution. Independent exploration tends to lead to low-quality solutions for most individuals, even if there is a high chance of some single solver finding the optimum (9). Social influence can result in a premature consensus on a good solution before the optimum is found (17, 18). Alternating between independent exploration and social influence may have reduced the chances of both types of getting stuck for IT triads.

Among all three treatments, the greatest improvements in solution quality occurred in IT triads during social-influence rounds—even for leading players with no better solution to copy (Fig. 2). Improvement in the mean solution is not surprising, as low performers were able to copy higher performers on rounds with social influence. However, there was also greater improvement in the quality of the best solution in a triad on social-influence rounds than on rounds without social influence. Social influence is especially beneficial—even for leading players—when it follows independent exploration that generates more diversity.

Fig. 3 plots parts of the correct solution that the leading player could learn from (if they were visible). It shows the number of correct solution legs (legs that were part of the globally optimal solution) in leading players’ solutions versus the number of correct solution legs in other solutions that were not also part of the leading solution. Leading players in IT triads were exposed to more correct legs than leading players in CT. Of course, lagging solutions in NT triads had the most correct solution legs that were not part of leading solutions, but these were never visible to leading players to learn from. Fig. 4 shows that leading players in IT made their solutions more similar to those of their neighbors during social-influence rounds—apparently taking advantage of that beneficial diversity.

Fig. 3.

Possibility of leaders learning from others’ solutions by treatment: fitted values (LASSO) for number of correct legs in leading players’ solutions versus number of correct legs in other players’ solutions that are not present in the focal leading player’s solution. Labels indicate round numbers.

Fig. 4.

Evolution of leading players’ solutions: fitted values (LASSO) for number of solution legs newly matching neighbors’ solutions from the previous rounds (from either copying or independent convergence on the same answers) versus legs not present in any solutions from the last round.

In addition to the above results, we ran a second set of trials evaluating the effect of another realistic condition: including a “storage” feature, in which individuals were reminded of their own best solution previous to the current round and could load it with a single mouse click. Overall, storing a solver’s best solution produced results that were qualitatively similar to social influence: Relative to our first set of trials, adding storage substantially decreased exploration (the number of unique solutions was 0.748 times the number without storage for CT, 0.706 for IT, and 0.799 for NT; Poisson, P < 0.001 for all comparisons) but resulted in an improvement in mean performance [with storage, log(1+difference from optimal distance) was 0.303 higher in CT, P = 0.010; IT: 0.271, P = 0.020; NT: 0.237, P = 0.009].

The chance of finding the optimum solution is related to both mean performance (and thus the number of individuals with good solutions) and the level of exploration (thus the relative chance of improving from an already good solution). Because storage improved one precursor to finding the optimum but decreased the other, storage had different effects on the raw rate at which the different treatments found the optimum. Without storage, CT and IT had a high mean performance, and IT and NT had high exploration. Storage reduced exploration and thus eliminated a major source of high performance in IT and NT. However, storage also increased the mean, creating a simultaneous improvement for all treatments. Taken together, CT, IT, and NT triads found the optimum in 39.1, 39.3, and 38.1% of trials, respectively, representing an increase for CT but decreases for IT and NT.

To simplify, we can think of finding the optimum as most likely when a subject’s solution is “in range”—that is, having a solution that can be tweaked to result in an optimum solution—and the subject continues to explore from there. Table 2 shows the raw rates of in-range rounds for each treatment condition along with the rate of improvement from in-range rounds. Rounds are considered in range if the optimum solution has not been found by a member of the triad, and the subject’s current solution has 22 or 23 correct solution legs (it is impossible to have 24 correct solution legs without violating the rules of the TSP); the table presents this number as a fraction of all rounds. The rate of improvement is calculated as the fraction of in-range rounds from which the focal subject’s solution improved.

Interestingly, the effect of greater improvement by top performers in IT triads on rounds with social influence (Fig. 2) is greatly reduced with the storage feature. Without the greater diversity from high exploration during rounds with independent exploration, the interplay between social influence and independent exploration did not yield any substantial benefit.

Intermittent breaks in interaction improve collective intelligence. Being exposed to diverse answers boosts performance, even if the answers one sees are worse than one’s own. To achieve this performance boost within a triad, there is a requirement for both independent exploration (to generate diversity) and interaction (to allow social influence). Only IT triads without storage have the necessary conditions for this boost to top performance. In CT triads, leaders are exposed to others’ answers, but they are not as diverse as IT triads without storage on average due to limited exploration. In NT triads, leaders are not exposed to others’ answers at all.

Like constant access to others’ answers, when one’s own past answers can be stored such storage reduces the additional boost to performance by leaders within IT triads. For the interplay between independent exploration and social ties to be beneficial, there must be sufficient exploration during the independent phases of the problem-solving task to generate diverse solutions that lead to learning. Storage works directly against this requirement by suppressing exploration and instead encouraging relative stasis at known solutions. Without the phase of exploration, we would not expect the overall performance to be substantially different from CT triads. Indeed, the coefficients in Table 1 show broad convergence between IT and CT when storage is present.

By shaping subjects’ behavior to take advantage of both independent exploration and social learning, intermittent interaction caused subjects to perform better on our complex problem-solving task. That implies, however, that task type represents a likely boundary condition for our results. In tasks where exploration or learning is unnecessary or impossible, we do not expect our results to hold. For example, pure coordination tasks [also known as “additive” tasks (32)], in which the quantity of distinct solutions or contributions is more important than their quality, would not necessarily reward learning. Similarly, some problem spaces are simple or “smooth” and do not require or reward extensive exploration. At the other extreme, other problem spaces may be so rugged that even arbitrarily similar solutions can be dissimilar in their quality; for such problems it would not be helpful to borrow and adapt part of a neighbor’s solution.

Our results suggest new avenues for research on the importance of interaction frequency for performance. For example, how does optimal frequency change with problem complexity, social network structure, the type of outcome sought, or the baseline collective intelligence factor of the group (1)? Might our results be moderated by different forms of interaction [such as the active consensus-oriented deliberation used in the second phase of the “hybrid structure” in the brainstorming literature (30)] or different approaches to using storage? Finally, might frequency of interaction differentially affect the various component mechanisms of social influence [e.g., free riding, evaluation apprehension, and production blocking (33)]? In short, our study suggests the importance of refocusing future research on the frequency and pattern of interaction, rather than its absence or presence.

Our main manipulation (NT, IT, or CT with storage off) reveals that intermittently present social influence achieves the beneficial aspects of both constant social influence and independence when searching complex solution spaces. Prior results showing the benefits of social influence in “wisdom of the crowd” tasks (15, 16) are due to less-confident low performers revising their solutions toward the mean after peer influence. Our results show something more: Triads find the optimum more and high performers do even better with intermittent ties, suggesting the presence of beneficial social learning for all participants, not just low performers. Indeed, intermittent social influence may mitigate the dangers inherent in both independent exploration (spending time on poor solutions) and social influence (premature consensus). Importantly, although past laboratory experimental work has focused on constant structures of social influence, real online and offline social ties are intermittent (29, 34), like our top-performing treatment.

In general this is a reassuring finding about collective intelligence in the wild but raises many questions about the design of always-on technologies that support collaborative and crowd work. Broadly speaking, productivity tools encourage people to build off of their own previous best work, and transparency-enhancing collaboration and networking tools encourage people to be in constant contact with one another. Extrapolating from our results, one could say that such technology use increases mean performance but depresses maximum performance in complex problem solving. Although much is gained from keeping people connected, even greater problem-solving performance could be achieved by redesigning technologies to intermittently turn on and off the influence that people feel from social ties and their own previous work.