Threat_Hunting_with_better_DataVisualization (BSides Brisbane 2025 version)

🦅 🐜 Threat Hunting with better Data Visualization Tatsuya Hasegawa (T.H) Threat Hunter (T.H) 1

Tatsuya Hasegawa • self-employed Threat Hunter for over 4y, total career 15y in cyber security. • Data Visualization Developer • OSS msticpy contributor • ISACA Nagoya Chapter Director • Security Research Subcommittee Chair • SNS (HN: hackeT, X: @T_8ase) • Certifications • CISSP, CISA, GSP#414, GIAC Advisory Board • GX-FA, GX-FE, GX-IH, GXPN,GPEN,GMOB,GREM,GCIH,GCFA,GCFE,GNFA, 2

My Custom Visualizations ( D3 series ) https://splunkbase.splunk.com/apps?author=hacket https://github.com/microsoft/msticpy/releases/tag/v2.15.0 3

• https://splunkbase.splunk.com/apps?author=hacket
• https://github.com/microsoft/msticpy/releases/tag/v2.15.0

My Custom Visualizations ( Vega series ) 4

Outline 1. Relationship between Threat Hunting and Data Visualization 2. What does it mean to fully understand your data? 3. Dashboard design challenges and practical tips 4. Introducing charts to dig deeper faster 5. Limitations of data visualization in current generative AI 6. Magic of custom visualization 5

Outline 1. Relationship between Threat Hunting and Data Visualization 2. What does it mean to fully understand your data? 3. Dashboard design challenges and practical tips 4. Introducing charts to dig deeper faster 5. Limitations of data visualization in current generative AI 6. Magic of custom visualization 6

What is Threat Hunting? Security products can NOT detect all threats => potential risk/threat of FN But having productʼs telemetries or logs!✨ I can.. Proactive detection, then respond quickly before risk spreads Even if you go hunting, there may not be any threats. 😅 Your findings sometimes havenʼt enough risks to report your organization. 😞 Even with the same analytical perspective, new discoveries if the timing is different. Threat hunting should be incorporated into daily operations. Similar to security monitoring? 🧐 Þ Yes, but security monitoring 👉 “fixed-point observation” (定点観測) Þ While, threat hunting 👉 “dynamic observation for weak area” 7

Threat Hunting approach - Hypotheses cards • Intelligence-Driven Hypo • Known evil, malicious similarity • ♦ IoC (Indicator of Compromise) • ❤ IoA (Indicator of Attack) in progress • ♣ Situation Awareness Hypo • Baseline, deviation from benign ♦ Intelligence -Driven (IoC) ❤ ♠ Intelligence Domain Expertise -Driven (IoA) • ♠ Domain Expertise Hypo • Hunterʼs past experience • Difficult to document and share • Cognitive bias? But valuable resource SANS “Generating Hypotheses for Successful Threat Hunting” 2016 https://www.sans.org/white-papers/37172/ ♣ Situation Awareness 8

• https://www.sans.org/white-papers/37172/

Hasegawa-style Drill Hunting Based on EDA (Exploratory Data Analysis) Instead of direct excavator, hands-on approach, drilling down layer by layer. Scoping widely at first、then dig deeper with more knowledge Noise Noise FP FP FP Dig! Dig! Dig! with understanding Noise Security Product Area FP FP Security Product Area Dig Wide at First 9

What is Data Visualization ? • The art of turning data into visual context • What would happen if with it? • Facilitate data understanding and speed up decision-making • Efficient information sharing • Discover patterns and trends Promotion of data understanding Operational Efficiency Total Solar Eclipse (皆既⽇⾷) Orbit Visualization (2017-2080) by Denise Lu 10 https://www.washingtonpost.com/graphics/national/eclipse/

• https://www.washingtonpost.com/graphics/national/eclipse/

Relation between Hunting and Visualization My Hunting Process Data source Event Category Point of View A Point of View B Suspicious Event Term Bird’s-eye Insect’s eye Visualization Visualization 🦅 🐜 Deep Analysis Search 11

How to move from Bird to Insect, no silver bullet! Look! Look! Inspect! 🦅 Bird eye Data cut by 1 or 2 features/fields Horizon Normal events Bubble Understood data enough Narrow down while excluding normal/benign Then, drill down! Insect eye 🐜 Anomaly Look at narrowed down with many features/fields Sunburst Sankey Multi-Dim Plot 12 Some icons from elastic

Outline 1. Relationship between Threat Hunting and Data Visualization 2. What does it mean to fully understand your data? 3. Dashboard design challenges and practical tips 4. Introducing charts to dig deeper faster 5. Limitations of data visualization in current generative AI 6. Magic of custom visualization 13

Understanding data enough for Threat Hunting • Most data for Threat Hunting => “Event” data • i.e. Data with a time axis and continue to grow • Understand all the data -> Look at all the data • Probably, impossible in enterprise !! • Key = Data Aggregation • Divide the data into parts and summarize the majority of the data Source • How to divide? Preprocessing 1. Data source filtering 2. Term (Time span) 3. Event category filtering Term Category 14

Preprocessing on my hunting for data understanding Events Time series aggregated Dictionary with datetime for referral Bucket events with datetime Lookup table Long term over 1mon period Short term Multivariate 1h~1w period analysis Anomaly Detection 15

Advice for boundary whether you understand the data enough 1. Data Source • Choose by your hunting scope 2. Term • View event data for at least the past six months (half of year) • 1 year for business which the events changes seasonally 3. Event Category 1. 2. Check rare values by starting with the field having smallest standard deviation In Splunk, “fieldsummary” command is useful. 4. Aggregation 1. 2. For IP addresses and port numbers, group by range and attribution type Reduce the data noise by excluding device and system-related events Do NOT miss the changes of event data structure by system updating !! e.g. New fields, New category values Monitor the change of data structure itself 16

Outline 1. Relationship between Threat Hunting and Data Visualization 2. What does it mean to fully understand your data? 3. Dashboard design challenges and practical tips 4. Introducing charts to dig deeper faster 5. Limitations of data visualization in current generative AI 6. Magic of custom visualization 17

What is SIEM dashboard used for? MUST Security Monitoring CAN SIEM Forensics Dash CAN Dashboard from ü App built-in ü Develop by ourselves Threat Hunting Board CAN SOAR 18

General SIEM Dashboards help our monitoring + World Maps Source: TOP 25 KIBANA DASHBOARD EXAMPLES https://logit.io/blog/post/the-top-kibana-dashboards-and-visualisations/ 19

But.. not enough for hunting • General SIEM Dashboard are for security monitoring. • During yearly, monthly, weekly and even daily monitoring, the information is summarized in a large format so that any operator can access the same information and make the same observations. 💣High missing possibility for the threat due to less information 🙃 In my experience, any anomalies found in this “large-grained” dashboard are either already triggering alerts from security devices or are false positives. Filtering Paradox Even if you can fully understand what remained after filtering, can you understand what disappeared after superficial filtering? 20

Better viz dashboard ideas/tips for hunting Inline Documentation Keep track of how much preprocessing has been done Use panel’s title and meta data area Graph No use Pie chart without time info Line Chart & Network Node Graph Drill down search is also useful. Table Keep time information even when stacking (aggregation) Create Summary column with 4W <when> <who> <what> <where> Highlight with icons ✅❌💀🔥 etc. 21

Outline 1. Relationship between Threat Hunting and Data Visualization 2. What does it mean to fully understand your data? 3. Dashboard design challenges and practical tips 4. Introducing charts to dig deeper faster 5. Limitations of data visualization in current generative AI 6. Magic of custom visualization 22

Favorite charts for hunting • Bird-eye 🦅 • Line Chart • Network Node Graph with line thickness and color bind • Single Value Metrics for catching error or abnormal • Insect-eye 🐜 • Table chart sophisticated with summary and categorical color bind • No need 3D, No need dynamic move => Make your eye more exhausted 💦👁 👁💦 23

Example of viz hunting dashboard (1)🦅 DNS-based Data Exfiltration 👇 Easy Hunt ! Demo data: Splunk BOTSv2 datasets (tz: JST utc+9h) https://github.com/splunk/botsv2 24

Example of viz hunting dashboard (2)🦅 Demo data: Splunk BOTSv2 datasets (tz: JST utc+9h) https://github.com/splunk/botsv2 25

Example of viz hunting dashboard (3)🐜 👈 Preprocessing documentation (filtering) Spike timing is different from each “src_ip” machine, suspicious of lateral movement.🧐 Demo data: Splunk BOTSv2 datasets (tz: JST utc+9h) https://github.com/splunk/botsv2 26

Example of viz hunting dashboard (4)🐜 👇 Drill down Umm🧐, this isnʼt on 08-24, but just before DNS-based Data Exfiltration on 08-26! ☞ lack of logging.. the event log was deleted? or 08-24 was more stealth behavior? ⚠FORENSIC Demo data: Splunk BOTSv2 datasets (tz: JST utc+9h) https://github.com/splunk/botsv2 27

Network Forensic attempt for 08-24 Demo data: Splunk BOTSv2 datasets (tz: JST utc+9h) https://github.com/splunk/botsv2 💾 Download some attack tools via FTP ☞ Congratulations🎉 Hunting complete on Weaponization phase!! 28

Outline 1. Relationship between Threat Hunting and Data Visualization 2. What does it mean to fully understand your data? 3. Dashboard design challenges and practical tips 4. Introducing charts to dig deeper faster 5. Limitations of data visualization in current generative AI THIS INFORMATION IS BASED ON THE VERSION AT JULY 2025 6. Magic of custom visualization 29

Limitations of Visualization with GenAI Data source: easy small Azure NetFlow data https://raw.githubusercontent.com/microsoft/msticpy/refs/heads/main/docs/notebooks/data/az_net_flows.csv Prompt: 「You are a professional threat hunter who detects anomalies in this Azure Netflow log. Detect anomalies and visualize the abnormal areas with appropriate charts and graphs. I will leave the type of chart and the granularity of visualization to you.」 OpenAI ChatGPT Deep Research (o3) • Ref: https://help.openai.com/en/articles/8437071-data-analysis-with-chatgpt • Chart by Matplotlib, Network Node Graph hasn’t been supported yet. • Cannot be emphasized automatically, needed a specific definition by prompt • Tends to use traditional methods from the past • Data conversion (preprocessing) looks black box without asking, and concerns about the accuracy Microsoft Data Formulator (2023~) • Ref: https://www.microsoft.com/en-us/research/blog/data-formulator-a-concept-driven-ai-powered-approachto-data-visualization/ • GenAI-powered data visualization tool (OSS), help us to transform data (preprocessing) • Chart by Vega-Lite, many special charts haven’t been supported yet • Concerns about the accuracy of data conversion • Depends on LLM model brain linked by API 30

GenAI can't do it alone, still needs the collaborate with us Data Formulator can check, modify charts and analyze more deeper interactively and sequentially by “Data Threads” feature, “with singleness of purpose” (⼆⼈三脚) 31

Outline 1. Relationship between Threat Hunting and Data Visualization 2. What does it mean to fully understand your data? 3. Dashboard design challenges and practical tips 4. Introducing charts to dig deeper faster 5. Limitations of data visualization in current generative AI 6. Magic of custom visualization 32

Magic of Custom Visualization different visualization methods with the same data on Splunk Custom Viz can modify - Layout - Color palettes and Transparency - Font and Size - Tool tip, bread crumb, drill down options “AS YOU WANT” 33

Magic of Custom Visualization (2) Can attract audience with the unfamiliar graph🤩 More interest at your analysis process😳 Not necessary for hunting😅 Customizing needs design sense🧐 Need description what this is👄 34

🦅 Now, Wrap Up! üPoint to utilize visualization for hunting 🐜 ü🦅 Bird-eye by Graphs Threat Hunter ü🐜 Insect-eye by sophisticated Table Chart ü[Baby Step] Your Insects are put into positions after your birds see the whole picture. üDo NOT recommend just applying the SIEM monitoring dashboards to Threat Hunting! Letʼs customize to go down a level of granularity. üAlong with describing what part of the data in whole is being shown there. üRemain easy documentation like “Jupyter Notebook” in your dashboard üCustom Visualization may have a worth for illustrating your hunting process 35

Thank you! This is my LinkedIn account QR. 36