Applied Research
Despite the advantages to leveraging the Azure AI Marketplace for some use cases, there are many scenarios in which performing Fine-Tuning, Deep Learning and MLops on your local hardware is preferable to performing the same in the cloud. In my experience, this often includes one or more of the following: Now, my prediction is that…
Read More “Deploying Locally Trained LLM & ML Assets to Azure” »
Retrieval Augmented Generation(RAG) is the method of choice for increasing the output accuracy of Natural Language Processing(NLP), generation-based models. I’m not claiming that this will eliminate your generative model’s chances to hallucinate (as some do), but it does push the the model to retrieve relevant documents or pieces of information from your own corpus, based…
Though most Large Language Models(LLM) feature strong out-of-the-box capabilities with a wide spectrum of knowledge, they often fall short when asked to perform in-depth tasks within a specific subject matter — making them less that useful within specialized domains, like cyber security. There are two basic options to remedy this shortcoming while mitigating sacrifice of…
In the previous article (The Big Data SIEM: Ingestion Layer with Azure) we establish a baseline of principles with which to ingest data at the multiple petabyte per day scale. Though many aspects align to lessons learned in review of Azure’s Well Architected Framework(WAF), we include our recommendations for API client calls, data transformations, team…
A common concern while exploring new architectural designs, concepts or technologies, is sourcing data which is similar enough to the live data you will be ingesting. Though there are tons of dummy datasets out there, they generally lack the specificity, depth or volume required for comprehensive testing of enterprise scale solutions — namely at the…
Read More “Babel — A High Performance Dummy Data Load Generator” »
Back in June 2024, I wrote a post about a homelab project I had started, which aimed to characterize the dark web. The project was ultimately a success, achieving an average of ~75k .onion v3 resources located (verified & enriched) per 24 hours, over a 6 day “final” trial period. This was roughly 10x the…
In our previous article (The Big Data SIEM), we opened the series with a tongue and cheek scenario, a detailed use-case and general overview of the basic four layers to the big-data SIEM as we (Tyson Barber and Tyler G.) like to frame them in the minds of decision makers on these types of projects….
You’ve just gotten into the office on a Monday and your CISO has pulled you into a meeting. The board has decided you’re moving your SIEM off of Splunk (Elastic, Rapid7, etc.) to save money…
A homelab project in Dark Web content discovery, behavioral traffic analysis, and database development.
A case study and recommended Azure cloud architecture, designed to enrich data and train machine learning models with industrial IoT production data.
