Mapping the Hidden Architecture of Your Organization: Using ML and Digital Exhaust to Decode Organizational Topology

How machine learning can transform the invisible patterns in your company’s communications into actionable insights about collaboration, innovation, and dysfunction

If you found this useful, you can support my independent research here: https://www.paypal.biz/vector1research

Your org chart is a beautiful lie.

It shows clean hierarchies, neat reporting lines, and logical divisions of labor. But anyone who’s actually worked in an organization knows the truth: real work happens through invisible networks of relationships, informal coalitions, and spontaneous collaborations that no org chart could ever capture.

What if you could see the real structure of your organization? What if you could map the actual pathways through which knowledge flows, identify the hidden connectors who keep teams functioning, spot the emerging silos before they become entrenched, and measure where innovation actually happens?

This is no longer a thought experiment. By applying machine learning to the digital exhaust your organization already generates — Teams messages, Slack channels, email metadata, document collaboration patterns, meeting attendance, code commits, and calendar overlaps — we can construct a dynamic, living map of organizational topology that reveals what traditional org charts never could.

Part I: The Data Foundation

The Rise of Organizational Network Analysis

Organizational Network Analysis (ONA) isn’t new. Researchers have studied informal networks in companies for decades. What’s changed is the scale, granularity, and analytical power we can now bring to bear.

Every day, your organization generates terabytes of interaction data:

• Communication metadata: Who emails whom, who messages whom, response times, thread depths
• Collaboration signals: Co-authorship on documents, simultaneous editing, shared file access
• Meeting patterns: Who attends together, who organizes, who’s optional vs. required
• Code and project artifacts: Commit patterns, review relationships, dependency structures
• Knowledge artifacts: Wiki contributions, documentation ownership, question-answering patterns

This “digital exhaust” — the byproduct of normal work — contains an extraordinarily rich signal about how your organization actually functions. The challenge is extracting meaningful patterns from the noise.

Data Sources: A Technical Inventory

Before modeling, you need data. Here’s what’s typically available and what each source reveals:

Communication Platforms

Microsoft Teams / Slack

• Direct messages (sender, recipient, timestamp, channel)
• Channel participation and activity levels
• Reaction patterns (who reacts to whose messages)
• Thread structure (who responds to whom)
• @mentions (explicit attention direction)
• Message content (for NLP analysis)

Email (Exchange, Gmail)

• Sender/recipient metadata
• CC/BCC patterns (visibility networks)
• Reply chains and response times
• Subject line clustering
• Attachment sharing patterns

Key derived features:

• Communication frequency matrices
• Response time distributions
• Reciprocity scores
• Time-of-day patterns
• Channel/topic affinity

Collaboration Platforms

Document Systems (SharePoint, Google Drive, Confluence)

• Co-authorship (multiple editors on same document)
• Comment/suggestion interactions
• Document sharing graphs
• Edit temporal patterns
• Content similarity between authors

Code Repositories (GitHub, GitLab, Bitbucket)

• Commit co-occurrence on same files
• Pull request review relationships
• Code review comment patterns
• Merge/approval hierarchies
• Codeowner overlaps

Project Management (Jira, Asana, Monday)

• Task assignment patterns
• Handoff relationships
• Blocker/dependency chains
• Sprint/project team composition over time

Calendar and Meeting Data

Calendar Systems

• Meeting co-attendance matrices
• Organizer/attendee power dynamics
• Meeting size distributions
• Recurring vs. ad-hoc patterns
• Calendar block patterns (availability signaling)

HR and Organizational Data

HRIS Systems

• Formal reporting relationships
• Team/department membership
• Tenure and role history
• Location and time zone
• Job level/seniority

This becomes the ground truth for comparing formal vs. informal structure.

Part II: Graph Construction and Representation

Building the Organizational Graph

The foundation of this analysis is constructing a graph representation of your organization. In this model:

Nodes represent entities:

• Individuals (primary)
• Teams/departments
• Projects
• Knowledge domains/topics
• Communication channels
• Documents and artifacts

Edges represent relationships:

• Direct communication (weighted by frequency, recency, reciprocity)
• Collaboration events (co-authorship, joint meetings, shared projects)
• Knowledge transfer (mentoring signals, Q&A patterns)
• Organizational proximity (same team, shared manager, physical location)

Edge Weight Calculation: The Critical Detail

Not all interactions are equal. Edge weights must capture relationship strength meaningfully. Here’s a robust weighting framework:

Communication Edge Weights

Basic frequency weighting:

w_ij = count(messages from i to j) + count(messages from j to i)

This is naive. Better approaches incorporate:

Recency decay:

w_ij = Σ exp(-λ * (t_now - t_interaction))

Where λ controls decay rate. Recent interactions matter more.

Reciprocity adjustment:

reciprocity_ij = min(out_ij, in_ij) / max(out_ij, in_ij)
w_adjusted = w_ij * (0.5 + 0.5 * reciprocity_ij)

One-way communication is weaker than mutual exchange.

Response time signal:

responsiveness_ij = 1 / (1 + median_response_time_ij)
w_adjusted *= (0.7 + 0.3 * responsiveness_ij)

Fast responses indicate stronger relationships.

Channel normalization: Different channels have different baseline frequencies. Normalize within channel before combining:

w_normalized = (w_raw - μ_channel) / σ_channel

Collaboration Edge Weights

Co-authorship strength:

w_coauthor = Σ (1 / n_authors_doc) for each shared document

Documents with fewer authors indicate stronger collaboration signal.

Meeting co-attendance:

w_meeting = Σ (1 / sqrt(meeting_size)) * meeting_duration

Small meetings indicate stronger intentional collaboration.

Code collaboration:

w_code = commits_same_files + 2 * (reviews_given + reviews_received)

Code review is higher-signal than incidental file overlap.

Multi-layer Graph Fusion

Organizations have multiple relationship types. Options for combining:

Additive fusion:

W_combined = α₁W_comm + α₂W_collab + α₃W_meeting

Where α weights are tuned or learned.

Multiplicative fusion (for high-confidence edges):

W_combined = W_comm^α₁ * W_collab^α₂ * W_meeting^α₃

Only strong when multiple signals agree.

Multi-layer preservation: Keep layers separate for analysis, combine only for specific metrics. This preserves richer information.

Temporal Graph Representation

Organizations change. Static graphs miss dynamics. Temporal representations:

Snapshot sequences:

G = [G_t1, G_t2, ..., G_tn]

Weekly or monthly snapshots enable trend analysis.

Continuous-time dynamic graphs:

G(t) with edges as (i, j, t_start, t_end, weight_function(t))

More complex but captures formation/decay.

Temporal edge features:

edge_ij_features = [
    creation_time,
    last_interaction_time,
    interaction_count_windows[7d, 30d, 90d, 365d],
    trend_slope,
    burstiness_score
]

Part III: Core Network Metrics and Their Interpretation

Centrality Measures: Who Matters and How

Centrality metrics quantify node importance, but different centralities capture different kinds of importance.

Degree Centrality

Definition: Number of connections a node has.

C_degree(i) = Σ_j A_ij  (unweighted)
C_degree(i) = Σ_j W_ij  (weighted)

Interpretation in organizational context:

• High degree: Person communicates with many others
• In-degree vs out-degree: Receivers vs. initiators
• Weighted vs unweighted: Volume vs. breadth

What it reveals:

• Communication hubs
• Highly connected individuals
• Potential overload candidates

Limitations:

• Doesn’t distinguish between connecting to important vs. unimportant nodes
• Chattiness ≠ importance

Output example:

Top 10 by Weighted Degree Centrality:
1. Sarah Chen (Engineering)      - 847.3  [Hub: connects 12 teams]
2. Marcus Johnson (Product)      - 723.1  [Bridge: Eng-Product]
3. Jennifer Walsh (CEO Office)   - 698.4  [Formal authority]
...

Betweenness Centrality

Definition: Fraction of shortest paths passing through a node.

C_betweenness(i) = Σ_{s≠i≠t} (σ_st(i) / σ_st)

Where σ_st is the number of shortest paths from s to t, and σ_st(i) is paths through i.

Interpretation in organizational context:

• High betweenness: Brokers, bridges, gatekeepers
• Controls information flow between groups
• Removal would disconnect parts of the organization

What it reveals:

• Cross-functional connectors
• Potential bottlenecks
• Single points of failure
• Informal influencers

Critical insight: High betweenness with low formal authority = hidden organizational linchpin. These people are often undervalued and overworked.

Output example:

Betweenness Centrality Analysis:

HIGH BETWEENNESS / HIGH AUTHORITY (Expected):
- CTO, VP Engineering, Head of Product

HIGH BETWEENNESS / LOW AUTHORITY (Hidden linchpins - RISK):
- Alex Rivera (Senior Engineer) - Bridges ML and Backend teams
- Priya Sharma (Product Manager) - Only PM who talks to Data Science
- Tom Wilson (Tech Writer) - Documentation flows through him

LOW BETWEENNESS / HIGH AUTHORITY (Potential disconnect):
- CFO (isolated from operational teams)
- VP Sales (siloed from product development)

Eigenvector Centrality

Definition: Importance based on connections to other important nodes.

C_eigen(i) = (1/λ) Σ_j A_ij * C_eigen(j)

Recursive definition: you’re important if important people connect to you.

Interpretation in organizational context:

• Captures influence through network position
• High score = connected to well-connected people
• “It’s not what you know, it’s who you know”

What it reveals:

• Core vs. periphery positioning
• Access to organizational power
• Information advantage

Variant — PageRank:

PR(i) = (1-d)/N + d * Σ_j (A_ji * PR(j) / out_degree(j))

Adds damping factor, more robust to sinks.

Closeness Centrality

Definition: Average shortest path distance to all other nodes.

C_closeness(i) = (N-1) / Σ_j d(i,j)

Interpretation in organizational context:

• How quickly can someone reach everyone else?
• Low closeness = peripheral, takes many hops to reach center
• High closeness = can efficiently broadcast/gather information

What it reveals:

• Information propagation speed from each node
• Organizational reachability
• Isolation patterns

Structural Position Metrics

Brokerage and Constraint

Burt’s Structural Holes Theory: Value comes from bridging disconnected groups.

Constraint measure:

C_constraint(i) = Σ_j (p_ij + Σ_k p_ik * p_kj)²

Where p_ij is the proportion of i’s network invested in j.

High constraint: Your contacts know each other (redundant network) Low constraint: Your contacts don’t know each other (you bridge structural holes)

Interpretation:

• Low constraint = high brokerage potential
• Access to diverse, non-redundant information
• Innovation correlates with low constraint

Output example:

Structural Holes Analysis:

EFFECTIVE BROKERS (Low Constraint, High Betweenness):
┌─────────────────────┬────────────┬─────────────┬──────────────────────────┐
│ Person              │ Constraint │ Betweenness │ Groups Bridged           │
├─────────────────────┼────────────┼─────────────┼──────────────────────────┤
│ Alex Rivera         │ 0.12       │ 0.089       │ ML, Backend, DevOps      │
│ Maria Santos        │ 0.15       │ 0.076       │ Design, Eng, Research    │
│ James Lee           │ 0.18       │ 0.065       │ Sales, Product, Customer │
└─────────────────────┴────────────┴─────────────┴──────────────────────────┘

CONSTRAINED HUBS (High Degree but High Constraint):
- Team leads with dense intra-team networks but few external connections
- Risk: team becomes siloed

Core-Periphery Structure

k-core decomposition: Recursively remove nodes with degree < k.

k-core(G):
    while exists node with degree < k:
        remove node
    return remaining subgraph

Coreness: Highest k for which node remains.

Interpretation:

• High coreness = core of organizational network
• Low coreness = periphery
• Useful for identifying truly central vs. superficially connected

Core-periphery model fitting: Fit idealized core-periphery structure, measure deviation.

Core nodes: densely connected to each other and periphery
Periphery nodes: connected to core but not each other

Output visualization:

Core-Periphery Structure:

CORE (coreness ≥ 8):
  ████████████████████████████
  │ Engineering Leadership    │
  │ Product Core Team         │
  │ Key Technical Architects  │
  ████████████████████████████

SEMI-PERIPHERY (coreness 4-7):
  ▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓
  │ Most individual contributors              │
  │ Junior-mid managers                       │
  │ Cross-functional project members          │
  ▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓

PERIPHERY (coreness ≤ 3):
  ░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░
  │ New hires                                        │
  │ Remote/isolated team members                     │
  │ Specialized roles with few collaborators         │
  │ Contractors                                      │
  ░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░

Part IV: Community Detection and Organizational Clustering

Why Community Detection Matters

Organizations naturally form clusters — teams, working groups, informal coalitions. Detecting these from interaction data reveals:

• Actual working groups vs. formal teams
• Cross-functional collaboration clusters
• Emerging silos
• Alignment between formal and informal structure

Algorithm Deep Dive

Louvain Method

Mechanism: Greedy modularity optimization with hierarchical aggregation.

Modularity definition:

Q = (1/2m) Σ_ij [A_ij - (k_i * k_j)/(2m)] * δ(c_i, c_j)

Measures density of edges within communities vs. expected by chance.

Algorithm:

1. Start with each node in its own community
2. For each node, calculate modularity gain from moving to neighbor’s community
3. Move node to community with maximum gain (if positive)
4. Repeat until no improvement
5. Aggregate communities into super-nodes, repeat

Strengths: Fast, scales well, hierarchical output Weaknesses: Resolution limit (misses small communities), non-deterministic

Output example:

Louvain Community Detection Results:

Detected 12 communities (modularity Q = 0.67)

Community 1 (n=45): "Engineering Core"
  - 89% Engineering department members
  - Key members: CTO, Principal Engineers
  - Internal density: 0.34
  - External connectivity: 0.08
  
Community 2 (n=32): "Product-Design Alliance"  
  - 67% Product, 28% Design, 5% Engineering
  - Bridges formal department boundaries
  - Key connector: VP Product
  
Community 3 (n=28): "ML Research Cluster"
  - Cross-cuts Engineering and Research
  - High external connectivity to academia
  - Emerging innovation hub
  
[...]

Infomap

Mechanism: Optimize description length of random walk on network.

Information-theoretic approach:

• Random walker on network
• Minimize bits needed to describe walk
• Communities = regions where walker stays longer

Map equation:

L(M) = q_↷ * H(Q) + Σ_i p_↻^i * H(P^i)

Strengths: Captures flow, handles directed networks, finds hierarchical structure Weaknesses: Computationally heavier

When to use: When information/work flows matter more than density.

Label Propagation

Mechanism: Nodes adopt most common label among neighbors.

Algorithm:

1. Assign unique label to each node
2. Randomly order nodes
3. Each node adopts majority label of neighbors
4. Repeat until convergence

Strengths: Very fast, no parameters Weaknesses: Non-deterministic, can produce trivial solutions

When to use: Large networks where speed matters, initial exploration.

Stochastic Block Models (SBM)

Mechanism: Generative probabilistic model of network structure.

Model:

• k latent communities
• P(edge between communities i and j) = B_ij
• Infer community assignments and connection probabilities

Hierarchical SBM: Allows nested community structure.

Strengths: Principled, can detect disassortative structure, provides uncertainty estimates Weaknesses: Computationally expensive, requires choosing k or using nested version

Output example:

Stochastic Block Model Results:

Inferred structure with 8 communities
Block matrix B (edge probabilities):

        C1    C2    C3    C4    C5    C6    C7    C8
C1     0.45  0.12  0.03  0.02  0.08  0.01  0.01  0.02
C2     0.12  0.38  0.15  0.04  0.02  0.03  0.01  0.01
C3     0.03  0.15  0.42  0.11  0.01  0.02  0.06  0.01
C4     0.02  0.04  0.11  0.35  0.02  0.01  0.02  0.03
...

Notable patterns:
- C2-C3 bridge: Product-Engineering interface
- C7 isolated: Remote team (intervention needed)
- C1 self-loop high: Engineering silo forming

Comparing Detected vs. Formal Structure

Critical analysis: How much does the informal network match the org chart?

Normalized Mutual Information (NMI):

NMI(C_detected, C_formal) ∈ [0, 1]

1 = perfect match, 0 = independent

Adjusted Rand Index (ARI):

ARI ∈ [-1, 1]

Measures pair-wise agreement, adjusted for chance.

Output example:

Formal vs. Informal Structure Comparison:

NMI Score: 0.58 (moderate alignment)
ARI Score: 0.45

Interpretation:
- Organization has significant informal structure diverging from org chart
- ~40% of detected communities span multiple formal departments

MISALIGNMENT ANALYSIS:

Formal "Engineering" splits into:
  - Backend Infrastructure (detected C1)
  - ML/AI Team (detected C3)
  - Frontend/Mobile (detected C5)
  → Consider whether formal structure reflects reality

Formal "Product" and "Design" merge into:
  - Product-Design Alliance (detected C2)
  → Strong cross-functional collaboration (positive)

Formal "Sales" is isolated:
  - Minimal detected connections to Product/Engineering
  → Potential silo risk (intervention needed)

Part V: Advanced ML Models for Organizational Topology

Graph Neural Networks for Organizational Representation

GNNs learn node representations that capture structural context.

Graph Convolutional Networks (GCN)

Layer definition:

H^(l+1) = σ(D̃^(-1/2) Ã D̃^(-1/2) H^(l) W^(l))

Where à = A + I (self-loops), D̃ = degree matrix of Ã.

Application to organizations:

• Input features: role, tenure, department, activity metrics
• Learn embeddings that capture structural position
• Downstream tasks: predict collaboration, detect anomalies, recommend connections

Implementation sketch:

import torch
from torch_geometric.nn import GCNConv

class OrgGCN(torch.nn.Module):
    def __init__(self, in_channels, hidden_channels, out_channels):
        super().__init__()
        self.conv1 = GCNConv(in_channels, hidden_channels)
        self.conv2 = GCNConv(hidden_channels, out_channels)
    
    def forward(self, x, edge_index, edge_weight):
        x = self.conv1(x, edge_index, edge_weight)
        x = torch.relu(x)
        x = self.conv2(x, edge_index, edge_weight)
        return x

# Node features: [tenure, role_embedding, dept_embedding, activity_level, ...]
# Output: learned organizational embeddings

Graph Attention Networks (GAT)

Mechanism: Learn attention weights for neighbor aggregation.

α_ij = softmax_j(LeakyReLU(a^T [Wh_i || Wh_j]))
h'_i = σ(Σ_j α_ij W h_j)

Advantage: Model can learn which relationships matter more.

Application: Attention weights reveal relationship importance:

High attention edges = strong influence relationships
Low attention edges = weak or redundant connections

Temporal Graph Networks

For dynamic organizational analysis:

Temporal attention:

Incorporate time encoding into attention mechanism
Recent interactions weighted higher

Applications:

• Predict relationship formation/decay
• Detect organizational change points
• Model onboarding integration trajectory

Node Embedding Methods

Node2Vec

Mechanism: Biased random walks + Skip-gram.

Parameters:

• p: return parameter (tendency to backtrack)
• q: in-out parameter (tendency to explore)

Walk strategy:

• Low p, high q: BFS-like (local structure)
• High p, low q: DFS-like (community structure)

Output: Dense vector for each node capturing structural similarity.

Application:

from node2vec import Node2Vec

# Create embeddings
node2vec = Node2Vec(G, dimensions=64, walk_length=30, 
                    num_walks=200, p=1, q=2)
model = node2vec.fit(window=10, min_count=1)

# Similar nodes = similar organizational position
similar_to_cto = model.wv.most_similar('CTO')
# Returns: other senior technical leaders, architects, etc.

Interpretation of embeddings:

• Cluster embeddings to find structurally similar roles
• Visualize with t-SNE/UMAP for organizational map
• Use as features for downstream prediction

Role2Vec / Struct2Vec

Focus: Structural role equivalence rather than proximity.

Concept: Nodes with similar local structure should have similar embeddings, even if distant in network.

Application: Find people playing similar organizational roles across different teams.

Example output:
Structurally equivalent groups:

Role Type "Team Hub":
  - Engineering: Alex, Sarah, Marcus
  - Product: Jennifer, David
  - Sales: Michelle
  Pattern: High degree, moderate betweenness, connects to manager

Role Type "Cross-Team Bridge":  
  - DevOps: Chris
  - Technical PM: Laura
  - Architect: James
  Pattern: Low in-team degree, high betweenness, connects clusters

Link Prediction for Collaboration

Predict missing or future edges.

Approaches

Heuristic scores:

• Common neighbors: |N(i) ∩ N(j)|
• Jaccard coefficient: |N(i) ∩ N(j)| / |N(i) ∪ N(j)|
• Adamic-Adar: Σ_{k ∈ N(i) ∩ N(j)} 1/log|N(k)|
• Preferential attachment: |N(i)| * |N(j)|

Learned predictors:

# Concatenate node embeddings, predict edge probability
def predict_link(embed_i, embed_j):
    combined = torch.cat([embed_i, embed_j, embed_i * embed_j])
    return sigmoid(mlp(combined))

Applications:

• Recommend introductions (“You should meet…”)
• Predict collaboration success
• Identify missing connections

Output example:

Recommended Connections (High predicted value, no current edge):

Priority 1 - Cross-functional bridges needed:
┌────────────────────┬────────────────────┬─────────┬────────────────────────┐
│ Person A           │ Person B           │ Score   │ Rationale              │
├────────────────────┼────────────────────┼─────────┼────────────────────────┤
│ ML Lead (Eng)      │ Data PM (Product)  │ 0.89    │ Shared project context │
│ DevOps Lead        │ Security Lead      │ 0.84    │ Structural hole        │
│ UX Researcher      │ Customer Success   │ 0.81    │ Customer insight flow  │
└────────────────────┴────────────────────┴─────────┴────────────────────────┘

Anomaly Detection in Organizational Graphs

Detect unusual patterns indicating problems.

Node Anomalies

Isolation anomaly: Node with unexpectedly low connectivity given role/tenure.

z_score = (degree_i - μ_role) / σ_role
If z_score < -2: flag as isolated

Overload anomaly: Node with unsustainably high connectivity.

If betweenness_percentile > 0.95 and role_level < senior:
    flag as potential burnout risk

Edge Anomalies

Unexpected connection: Edge between nodes that shouldn’t interact.

If P(edge | node features) < 0.01 and edge exists:
    flag for review

Missing expected connection: Predicted edge that doesn’t exist.

If P(edge | node features) > 0.9 and no edge:
    potential silo signal

Temporal Anomalies

Sudden connectivity change:

If |degree_t - degree_{t-1}| / degree_{t-1} > 0.5:
    flag as potential organizational issue

Relationship decay:

If edge_weight_t / edge_weight_{t-6mo} < 0.3:
    relationship deteriorating

Part VI: Content-Based Analysis with NLP

Topic Modeling for Knowledge Networks

Understanding what people discuss, not just that they communicate.

LDA (Latent Dirichlet Allocation)

Model:

• Documents (message threads, channels) as mixtures of topics
• Topics as distributions over words

Application:

• Map topics to people (who discusses what)
• Track topic evolution over time
• Identify knowledge domains

Output example:

Extracted Topics (k=20):

Topic 3: "Infrastructure & DevOps"
  Top terms: kubernetes, deployment, cluster, scaling, monitoring
  Top contributors: DevOps team, SRE, Platform Engineering
  Trend: Growing 15% QoQ
  
Topic 7: "Customer Issues"
  Top terms: customer, issue, support, ticket, escalation
  Top contributors: Support, Customer Success, Product
  Cross-team flow: Support → Product (high), Support → Engineering (low)
  Gap identified: Engineering disconnected from customer pain
  
Topic 12: "ML/AI Development"
  Top terms: model, training, inference, dataset, accuracy
  Top contributors: ML team, Data Science, Research
  Emerging connections: Recently bridging to Product

BERTopic

Modern alternative using transformers:

1. Generate document embeddings (BERT/Sentence-BERT)
2. Reduce dimensionality (UMAP)
3. Cluster (HDBSCAN)
4. Extract topic representations (c-TF-IDF)

Advantages:

• Better semantic understanding
• Handles short texts (messages) better
• Dynamic topic modeling built-in

Application:

from bertopic import BERTopic
from sentence_transformers import SentenceTransformer

# Custom embedding model for organizational domain
embedding_model = SentenceTransformer('all-mpnet-base-v2')

topic_model = BERTopic(
    embedding_model=embedding_model,
    nr_topics="auto",
    calculate_probabilities=True
)

topics, probs = topic_model.fit_transform(messages)

# Get topic evolution
topics_over_time = topic_model.topics_over_time(
    messages, timestamps
)

Expertise Inference

Problem: Who knows what? Map people to knowledge domains.

Approach 1: Topic-Author association

expertise(person, topic) = Σ P(topic | document) for documents by person

Normalized by total activity.

Approach 2: Question-Answer patterns

If person answers questions about topic:
    expertise_score += answer_quality * topic_relevance

Approach 3: Document authorship

expertise(person, topic) = Σ document_authority * topic_weight

Where document_authority = PageRank of documents.

Output example:

Expertise Map:

MACHINE LEARNING:
  ████████████████████░░░░  Sarah Chen (0.92)
  ███████████████░░░░░░░░░  Alex Rivera (0.78)
  ██████████░░░░░░░░░░░░░░  James Wu (0.54)
  
KUBERNETES/INFRASTRUCTURE:
  █████████████████████░░░  Chris Anderson (0.95)
  ████████████████░░░░░░░░  DevOps Team Lead (0.82)
  
CUSTOMER DOMAIN KNOWLEDGE:
  ████████████████████░░░░  Customer Success Lead (0.88)
  ██████████████░░░░░░░░░░  Solutions Architect (0.71)
  ████░░░░░░░░░░░░░░░░░░░░  Product Team (0.23)  ← GAP

Knowledge Risk Assessment:
- ML expertise concentrated (bus factor = 2)
- Kubernetes: good redundancy
- Customer knowledge: not reaching Engineering (silo)

Sentiment and Relationship Quality

Sentiment analysis on communications:

relationship_quality(i, j) = mean(sentiment(messages_ij))

Indicators:

• Positive sentiment = healthy collaboration
• Declining sentiment = relationship deterioration
• High variance = unstable relationship

Caution: Sentiment models can misread technical discussions. Domain adaptation needed.

Part VII: Temporal Analysis and Dynamics

Tracking Organizational Change

Metric Trajectories

Track key metrics over time:

centrality_trajectory(i) = [C(i, t1), C(i, t2), ..., C(i, tn)]
community_stability = NMI(communities_t, communities_{t-1})
global_clustering_coefficient_trend

Output example:

ORGANIZATIONAL DYNAMICS REPORT (Past 6 Months)

Network Density:
  Jan: 0.042 → Jun: 0.038 (-9.5%)
  ⚠️ Organization becoming more sparse
  
Community Stability:
  NMI(month-over-month): 0.89, 0.91, 0.72, 0.85, 0.88
  ⚠️ March disruption: Reorg detected in data before announcement
  
Key Individual Trajectories:
  
  Rising Influence:
  - New PM Lead: betweenness 0.02 → 0.08 (+300%)
  - ML Engineer promoted: eigenvector 0.15 → 0.31
  
  Declining Connectivity:
  - Senior Architect: degree 45 → 28 (-38%)
    → Disengagement signal? Check-in recommended
  - Remote Team Lead: closeness 0.32 → 0.21
    → Team becoming isolated

Change Point Detection

Methods:

• CUSUM (Cumulative Sum)
• PELT (Pruned Exact Linear Time)
• Bayesian change point detection

Application: Detect when organizational structure significantly shifts.

Detected Change Points:

March 15: Major restructuring
  - 3 communities merged
  - Betweenness centrality redistributed
  - Correlates with: Announced reorg (March 12)

August 3: Unexpected shift
  - New cluster forming around ML project
  - Cross-team edges increased 40%
  - No formal announcement: Organic collaboration emerging
  
October 20: Concerning pattern
  - Sales team connectivity dropped 25%
  - Internal density increased (silo forming)
  - Recommend intervention

Onboarding Integration Analysis

Track how new hires integrate.

Integration trajectory:

integration_score(t) = weighted_degree(t) / expected_degree_for_tenure

Healthy integration: Score approaches 1.0 within 3–6 months.

Output example:

NEW HIRE INTEGRATION ANALYSIS (Last 90 Days)

Well-Integrated:
  - Alex (Eng): 0.85 integration score at 60 days ✓
    Strong connections: Mentor, team, cross-functional
    
Slow Integration:
  - Sam (Product): 0.35 integration score at 60 days ⚠️
    Limited to: Direct team only
    Missing: Engineering connections (critical for role)
    Recommendation: Structured introductions needed
    
At Risk:
  - Jordan (Remote): 0.15 integration score at 90 days ⚠️⚠️
    Pattern: Declining week-over-week
    Connections: Manager only
    Action required: Immediate intervention

Part VIII: Multi-Modal Analysis and Fusion

Combining Data Sources

Different data sources reveal different aspects:

Source Reveals Limitations Email Formal communication, external contacts Declining usage, formal tone Slack/Teams Informal collaboration, real-time work Noisy, channel structure biases Meetings Intentional collaboration, decision-making High cost, scheduled only Documents Deep collaboration, knowledge creation Misses discussion Code Technical collaboration, dependencies Engineering only

Fusion Strategies

Feature-Level Fusion

Combine features before modeling:

edge_features = [
    email_frequency,
    chat_frequency,
    meeting_co_attendance,
    doc_co_authorship,
    code_collaboration
]

Train single model on combined features.

Decision-Level Fusion

Train separate models, combine predictions:

final_prediction = α₁ * email_model + α₂ * chat_model + α₃ * meeting_model

Learn weights or use equal weighting.

Graph-Level Fusion

Multiplex graph analysis:

G_multiplex = {G_email, G_chat, G_meeting, G_docs}

Analyze layers separately and together:

• Layer-specific metrics
• Cross-layer patterns (e.g., email but no chat = formal-only relationship)
• Aggregate metrics across layers

Cross-Modal Validation

Use one modality to validate insights from another:

CROSS-MODAL VALIDATION REPORT:

Finding: Engineering-Sales silo detected
  
Evidence by modality:
  Email: Low cross-boundary volume ✓
  Slack: No shared channels ✓
  Meetings: 0.5% meeting co-attendance ✓
  Documents: No shared docs ✓

Confidence: HIGH (all modalities agree)

---

Finding: PM-Engineering strong collaboration

Evidence by modality:
  Email: Moderate volume
  Slack: High volume in shared channels ✓
  Meetings: Frequent co-attendance ✓
  Documents: Shared PRDs ✓
  Code: Low (expected for PM role)
  
Confidence: HIGH

Part IX: Interpreting and Presenting Results

Organizational Health Dashboard

Key metrics for ongoing monitoring:

ORGANIZATIONAL HEALTH DASHBOARD

                          Current    Trend     Benchmark    Status
                          -------    -----     ---------    ------
Network Density            0.038     ↓ 5%      0.04-0.06    ⚠️ LOW
Clustering Coefficient     0.42      → 0%      0.3-0.5      ✓ OK
Avg Path Length            3.2       ↑ 8%      2.5-3.5      ⚠️ RISING
Modularity                 0.68      ↑ 12%     0.4-0.6      ⚠️ HIGH
Largest Component          94%       → 0%      >95%         ⚠️ FRAGMENTING
Core Size                  28%       ↓ 3%      25-35%       ✓ OK
Gini (Centrality)          0.61      ↑ 5%      <0.5         ⚠️ UNEQUAL

INTERPRETATION:
- Organization becoming more siloed (modularity up)
- Information flow slowing (path length up)
- Connectivity increasingly unequal (Gini up)
- Recommend: Cross-team initiatives, bridge building

Visualization Approaches

Network Visualization

Force-directed layouts:

• Nodes = people, edges = relationships
• Color by department, size by centrality
• Works up to ~500 nodes

Hierarchical layouts:

• Respect formal org structure
• Overlay informal connections
• Highlight divergence

Adjacency matrices:

• Reordered by community
• Shows block structure clearly
• Scales better than node-link

Temporal Visualizations

Animation:

• Show network evolution over time
• Highlight emerging/dissolving clusters
• Powerful for presentations

Small multiples:

• Snapshots at intervals
• Enable comparison
• Better for analysis

Trajectory plots:

• Individual/team metrics over time
• Trend lines and anomalies
• Actionable insights

Reporting Framework

Executive summary structure:

1. OVERALL HEALTH ASSESSMENT
   - 3-5 key metrics with traffic lights
   - Trend direction and velocity
   - Comparison to benchmarks

2. KEY FINDINGS
   - 3-5 most important patterns
   - Evidence supporting each
   - Confidence level
   
3. RISK AREAS
   - Silos forming/formed
   - Single points of failure
   - Declining relationships
   - Integration failures
   
4. OPPORTUNITIES
   - Emerging collaboration hubs
   - Effective bridge-builders to recognize
   - Organic innovations to support
   
5. RECOMMENDATIONS
   - Specific interventions
   - Prioritized by impact and feasibility
   - Success metrics for each

Part X: Alternative and Complementary Methods

Survey-Based ONA

Traditional approach: Ask people about their networks.

Typical questions:

• Who do you go to for information?
• Who do you collaborate with regularly?
• Who energizes you?
• Who do you trust for important decisions?

Advantages:

• Captures perception and sentiment
• Identifies aspirational connections
• Reveals energy/trust dimensions not in digital data

Disadvantages:

• Recall bias
• Response burden
• Survey fatigue
• Point-in-time only

Hybrid approach: Use digital data as baseline, survey for validation and enrichment.

Ethnographic Methods

Deep qualitative understanding:

• Observation of work practices
• Interviews with key individuals
• Shadowing across roles
• Cultural artifact analysis

When valuable:

• Interpreting surprising quantitative findings
• Understanding why patterns exist
• Identifying invisible work
• Context for interventions

Simulation and Agent-Based Models

Model organizational dynamics forward:

Approach:

• Create agents representing individuals/teams
• Define interaction rules based on observed patterns
• Simulate evolution under different scenarios

Applications:

• Reorganization impact prediction
• Growth scenario planning
• Intervention testing

Example:

Simulation: "What if we merge Engineering and DevOps?"

Scenario A: Formal merge, no intervention
  - Predicted integration time: 8 months
  - Silo probability: 65%
  - Productivity dip: 15% for 4 months

Scenario B: Merge with structured integration program
  - Predicted integration time: 4 months  
  - Silo probability: 20%
  - Productivity dip: 8% for 2 months

Recommendation: Invest in integration program

Comparison with Traditional Methods

Method Coverage Frequency Depth Bias Cost Digital Exhaust ML Comprehensive Continuous Medium Behavioral Medium (setup) Survey ONA Sampled Periodic High Recall/Social High (ongoing) Interviews Selective One-time Very High Selection Very High Observation Limited Intensive Very High Observer Very High HR Data Only Comprehensive Static Low Formal only Low

Recommendation: Use digital exhaust as foundation, supplement with surveys for validation and qualitative methods for interpretation.

Part XI: Implementation Considerations

Privacy and Ethics

Critical concerns:

1. Surveillance vs. insight

• Focus on aggregate patterns, not individual monitoring
• Resist pressure to use for performance management
• Establish clear use policies

2. Consent and transparency

• Inform employees about data collection
• Explain how data will be used (and not used)
• Consider opt-out mechanisms

3. Data minimization

• Collect only what’s needed
• Aggregate quickly, delete raw data
• Limit access to identifiable data

4. Governance

• Ethics review for new analyses
• Employee representation in oversight
• Regular audits of actual use

Technical Architecture

┌─────────────────────────────────────────────────────────────────────┐
│                        DATA SOURCES                                  │
│  ┌──────────┐ ┌──────────┐ ┌──────────┐ ┌──────────┐ ┌──────────┐  │
│  │  Email   │ │  Slack/  │ │ Calendar │ │  Docs    │ │  HRIS    │  │
│  │  Server  │ │  Teams   │ │          │ │          │ │          │  │
│  └────┬─────┘ └────┬─────┘ └────┬─────┘ └────┬─────┘ └────┬─────┘  │
└───────┼────────────┼────────────┼────────────┼────────────┼────────┘
        │            │            │            │            │
        ▼            ▼            ▼            ▼            ▼
┌─────────────────────────────────────────────────────────────────────┐
│                      DATA INGESTION LAYER                            │
│  ┌──────────────────────────────────────────────────────────────┐   │
│  │  - Extract metadata (anonymize content where possible)        │   │
│  │  - Normalize timestamps, identifiers                          │   │
│  │  - Apply retention policies                                   │   │
│  │  - Privacy filtering                                          │   │
│  └──────────────────────────────────────────────────────────────┘   │
└─────────────────────────────────────────────────────────────────────┘
        │
        ▼
┌─────────────────────────────────────────────────────────────────────┐
│                     GRAPH CONSTRUCTION LAYER                         │
│  ┌──────────────────────────────────────────────────────────────┐   │
│  │  - Entity resolution                                          │   │
│  │  - Edge weight calculation                                    │   │
│  │  - Temporal graph updates                                     │   │
│  │  - Multi-layer graph storage                                  │   │
│  └──────────────────────────────────────────────────────────────┘   │
└─────────────────────────────────────────────────────────────────────┘
        │
        ▼
┌─────────────────────────────────────────────────────────────────────┐
│                       ANALYTICS LAYER                                │
│  ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐   │
│  │  Centrality │ │  Community  │ │  Temporal   │ │    NLP      │   │
│  │  Metrics    │ │  Detection  │ │  Analysis   │ │  Pipeline   │   │
│  └─────────────┘ └─────────────┘ └─────────────┘ └─────────────┘   │
│  ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐   │
│  │    GNN      │ │    Link     │ │   Anomaly   │ │  Simulation │   │
│  │  Embeddings │ │ Prediction  │ │  Detection  │ │   Engine    │   │
│  └─────────────┘ └─────────────┘ └─────────────┘ └─────────────┘   │
└─────────────────────────────────────────────────────────────────────┘
        │
        ▼
┌─────────────────────────────────────────────────────────────────────┐
│                      PRESENTATION LAYER                              │
│  ┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐        │
│  │   Dashboards    │ │   Reports       │ │   API           │        │
│  │   (Leadership)  │ │   (Analysts)    │ │   (Integration) │        │
│  └─────────────────┘ └─────────────────┘ └─────────────────┘        │
└─────────────────────────────────────────────────────────────────────┘

Tooling Ecosystem

Graph databases:

• Neo4j (most mature)
• Amazon Neptune
• TigerGraph

Graph analytics:

• NetworkX (Python, prototyping)
• graph-tool (Python, performant)
• igraph (R/Python)
• Spark GraphX (distributed)

GNN frameworks:

• PyTorch Geometric
• DGL (Deep Graph Library)
• Spektral (Keras)

Visualization:

• Gephi (standalone)
• D3.js (web)
• Pyvis (Python)
• Graphistry (GPU-accelerated)

NLP:

• spaCy, NLTK (traditional)
• Hugging Face Transformers
• BERTopic

Platforms (commercial):

• Microsoft Workplace Analytics
• Humanyze
• Polinode
• Cognitive Talent Solutions

Part XII: From Insight to Action

Intervention Framework

Translate findings into action:

FINDING → HYPOTHESIS → INTERVENTION → MEASUREMENT

Example:
FINDING: Engineering-Sales silo detected
HYPOTHESIS: Lack of structured touchpoints causes disconnect
INTERVENTION: Monthly Engineering-Sales sync, shared Slack channel
MEASUREMENT: 
  - Cross-boundary edge count (target: +50% in 6 months)
  - Meeting co-attendance (target: +25%)
  - Sales-reported engineering responsiveness (survey)

Prioritization Matrix

HIGH IMPACT
                          │
     QUICK WINS           │          STRATEGIC INITIATIVES
     • Connection         │          • Reorganization
       recommendations    │          • Major process changes
     • Meeting            │          • Platform changes
       optimization       │
     • Recognition of     │
       hidden linchpins   │
─────────────────────────┼─────────────────────────────────
                          │
     FILL-IN              │          COMPLEX PROBLEMS
     • Minor process      │          • Cultural change
       tweaks             │          • Deep-seated silos
     • Documentation      │          • Executive dysfunction
       improvements       │
                          │
                      LOW IMPACT

Success Metrics

Track intervention effectiveness:

Leading indicators (weeks):

• Edge count changes
• Response time improvements
• Meeting pattern shifts

Lagging indicators (months):

• Survey-measured collaboration quality
• Project outcome metrics
• Employee sentiment
• Retention patterns

Conclusion

The organizational graph is always there, hidden in the data exhaust of daily work. Machine learning gives us the tools to make it visible.

The organizations that thrive will be those that learn to read their own topology — to see the emerging silos before they calcify, to identify the hidden connectors before they burn out, to understand where innovation actually happens and where it’s blocked.

This isn’t about surveillance or control. It’s about understanding. It’s about making the invisible visible so that leaders can make better decisions, so that employees can navigate more effectively, so that organizations can become more than the sum of their parts.

The math is available. The data exists. The only question is whether we’ll use these tools wisely — in ways that respect privacy, that aggregate before individualizing, that focus on enabling rather than controlling, that make organizations more humane rather than less.

The future of organizational intelligence is a living map of how work actually happens. We now have the tools to build it.

Further Reading:

• Pentland, Alex. “Social Physics: How Good Ideas Spread”
• Cross, Rob & Parker, Andrew. “The Hidden Power of Social Networks”
• Burt, Ronald. “Structural Holes: The Social Structure of Competition”
• Newman, Mark. “Networks: An Introduction”
• Granovetter, Mark. “The Strength of Weak Ties”
• Hamilton, William. “Graph Representation Learning”
• Barabási, Albert-László. “Network Science”

If you’re working on organizational network analysis or considering implementation, I’d love to hear about your experiences. What patterns have you found? What challenges have you faced?

About the Author

Brian Curry is a Kansas City-based Data Scientist, AI expert, and founder of Vector1 Research, where he explores the frontier of Artificial Intelligence, Causal Inference, Agentic Systems, and Economic Modeling.

His work focuses on building intelligent systems that reveal the causal structures driving performance across marketing, content, and economics. He is the creator of PyCausalSim (causal discovery through simulation), Papilon (complex system optimization), Daedalus (causal and economic analysis), and Memory-Node Encapsulation (data structures for artificial episodic memory).

Contact: brian@vector1.ai