ESCALATE is coming. Here’s why it matters.

Share and earn Cybytes
Facebook Twitter LinkedIn Email

First, let’s start with a little history. In the summer of 2014, co-founders Evan and John started Point3 Security at the Bagels N’ Grinds coffee shop in Hanover, MD.  This was not the formation paperwork. This was the actual business.

We launched by teaching two and three-day seminars on niche topics like malware repurposing; design considerations for command and control; and binary reverse engineering. One of our brochures traveled home with a happy student and, shortly thereafter, we were “invited” (summoned) to the Pentagon for a meeting. We obliged, showing up in our T-shirts, ripped shorts, and flip flops among a sea of suits and ties.

That day, we learned that the military had not yet figured out how to train up its operational workforce. Like the private sector, the military was relying on a mix of college courses and industry certifications for the bulk of its curriculum. Then, bolting on a smattering of law, policy, and doctrine and, boom, they had whipped up a tailored training program (ha!).

The Pentagon said things like: By incorporating and emphasizing the CISSP [1], our operators (should) know that a “background check” – and not a “penetration test” – is an administrative security control that protects against insider attacks. (Boom! Take that, Russia!) Not sure which phase of a penetration test has the “scanning” part? Ask a CEH [2]. Rad! See you on the Nets, North Korea! [3] And if you have a Security+ cert, you know that Trojans can be inadvertently installed via a USB drive (as opposed to Spam, Buffer Overflow, and Logic Bomb) because one can meet Security+ Object 3.1 [4] without having to analyze malware.

We sat back in our chairs and surmised that the best the industry has come up with, to date, is a series of “my definitions are better than your definitions” knowledge-based (i.e. memorization-based) tests.  “Read the book and you’re fine. Don’t own a computer? Never touched a computer? Not actually a problem.” (Ugh).

Our contact at the Pentagon said they had an issue cultivating operational talent. After the first meeting, it was clear why. On the way home, we said to each other, “We’re teaching the wrong things. The U.S. is wholly unprepared to engage in a cyber war.” Damn, that was a powerful realization.


What we believe and what happened next.

We believe that U.S. adversaries aren’t going to be taken down by choosing ‘which of the following four options is the best regulation as it applies to a particular scenario.’  Adversaries are teams of smart, well-funded people utilizing indigenous hardware or software that’s not necessarily available for sale at Best Buy, is running in a foreign language, and is used on a variety of processor architectures. These tools and systems need to be assessed for vulnerabilities, with exploits that don’t fail or produce side-effects, and where some operational effect can occur (in many cases without being detected, even years later).

We believe that we need human talent that thrives in the proverbial Kobayashi Maru [5].  We need people who can work under-resourced through tiredness and bureaucratic adversity. The solution isn’t always something that can be copy/pasted from That is hard. That is the job.


No one is training for that job.

In our meeting with the Pentagon, they asked: “How would you make sure the same topics – scanning, exploitation, persistence, effects, and analyses – are covered in a manner that ensures a graduate would be immediately useful to a line level manager after the training?”

Back at the drawing board, we came up a shedding of the Victorian Era learning model – one that relies on rows and rows of students, canned lectures, and multiple-choice tests – for a return to tradecraft.

Enter the Cyber Operations Academy Course (COAC), which uses the cognitive apprenticeship learning model. Its curriculum is problem-based, and the learning principals are constructivist. Students are evaluated in teams. There are no lectures or slides and all problem solving is done on operationally relevant scenarios. Everyone meets up twice daily to help break the “I’m stuck, and I’ve exhausted all other known resources” patterns. If students say, “I know what to do next and it’s hard and will take time,” our response is, “Great, go take the time, and if you don’t know what to do next, we’re here for you.”

The pillars of cognitive apprenticeship demonstrate the ongoing, non-linear cycle of learning:


An interlude, if you’re so inclined, of Nerd Talk.

The very first challenge given to COAC students is to exploit a target and retrieve a file. The target software is vulnerable in some way to a remote attacker. It’s software we created, so inherently the problem is to discover a 0day vulnerability and author a 0day exploit. The target has DEP and ASLR enabled. The target service runs on a system hardened via a chroot environment with many ‘normal’ elements crippled to an attacker. The network is monitored. If you get caught you lose. This will take a month on average. Never do we give out the answer. Never do we say, “Time is up; hope you got it; let’s move on.”  Students work the problem to completion – even if it takes a month.

Although the following terms are part of the solution, we don’t use these terms in class – as they would be a distraction: ROP Gadgets, shellcode, memory carving and buffer overflow. The moment we give them a glossary of terms, it’s game over. We don’t talk about the Cyber Kill Chain. We don’t share pictures of the OSI Model. Yes, they are programming (x86 Assembly, C, and Python), they are scanning, and they are setting up mock networks to test prior to attacking the actual target. And you know what? The students are fine, even without the Kill Chain and OSI model!


The results so far.

We’ve been teaching COAC for three years and the results are staggering. After six months of immersion, alumni have placed in SANS’ Netwars [6], Raytheon’s Game of Pwns [7], and the Department of Energy’s CyberFIRE [8], to name a few.


But, don’t just take our word for it.

The course has been evaluated by the Institute for Defense Analyses (IDA) and Advanced Distributed Learning (ADL) initiative. These outfits sent us data scientists and PhDs in formative methods and learning theory. They observed and put together instruments measuring the curriculum, the instructors, and the students. They made maps of the steps required to detect and respond to a 0day breach, and to create and deploy a keylogger that doesn’t get flagged by Antivirus software.

There’s also a calculable unit that can be used to measure knowledge transfer called “Cohen’s d [9] [10]”.  It’s a 0-2 scale and participants in effective classes can expect a proficiency gain of 0.8 Cohen’s d. COAC students earn an average of 1.72 Cohen’s d on knowledge and 1.84 Cohen’s d on practical application of that knowledge.

Plus, every year, as a benchmarking exercise, COAC invites professional teams to compete in a 3-day Cyber A3 (All Against All) live-fire exercise. In the first year, a student team placed 2nd out of 8 teams. In the second year, 1st and 2nd place positions were held by the students, NOT the operational teams from the Department of Defense, Department of Homeland Security, and Industry (e.g. professional pen test teams).  In short, students with just six months of experience are holding their own against the pro teams.


The confirmation that cyber is blind.

One of the cool things about IDA’s and ADL’s data capture is that it shows NO correlation between one’s progression and one’s college degree status. In other words, folks without college experience did just as well as the Master’s holders. The same is true for Officer vs Enlisted, and of household income levels..

To a practitioner, this makes total sense. Cyber culture has always been about meritocracy. You can be a pierced, tattooed loner freak in the ‘real world’, and always have a welcome home within the cybersecurity community. There is no geographic boundary, no concept of race, gender or sexual orientation. If you’re a puzzle person and can prove it, you’re accepted.

Per the data, COAC’s problem-based curriculum would apply also to the private sector. In 2017, Mayor Rahm Emmanuel brought the COAC program to Chicago [11].  At the time of this writing, the course will wrap up in a few weeks. It does NOT bear a college degree or come with any college credit. However, it appears that its graduates will find 100% job placement at firms of varying shapes and sizes, cut across a wide set of vertical markets.

In the Chicago COAC program, the student body includes:

These students use debuggers and disassemblers daily. The have learned impressive soft skills in a short amount of time: how to work as a team; how to budget time and resources; and how to brief a solution to a non-technical audience.

The Economist Intelligence Unit cited the Mayor’s visionary partnership in its 2017 Safe Cities Index [12] ranking Chicago as the #1 cyber safe city in North America. Kinda neat!


The crystallization of needs.

As time passed on our journey, three central needs crystallized.

First – Identification. The Pentagon customer noted a problem not yet solved. When one joins the military, one goes through a battery of assessments called the ASVAB. This is a relatively quick instrument to match needs with talent. Years and years of incremental improvements enable the military to look at a candidate and say “That dude – infantry. Her? Artillery. Send him to intelligence and she’ll make a fine linguist.”

Yet, there is not a known good instrument for cyber. Presently, a mostly random smattering of personnel are assigned to go to cyber school, and if they don’t wash out – and the wash out rate can be as high as 50% for ENTRY level schools – they assume a good fit. This, as one can imagine, is quite costly.  To fill X slots, they need to send 2*X candidates, and since they don’t know which candidates to send, they send everyone. This process is super ineffective.

Second – Validation. Using our own company as an example, we’ve found that resumes are mostly useless and interviews only slightly less so. And since it’s hard to ascertain right fit, we must read every resume and interview every candidate. Ain’t nobody got time for that at a startup. So, how can companies like ours objectively assess talent and make the right hiring decisions?

Third – Cultivation. I recently had the opportunity to attend a conference on cyber workforce development. It was incredible. An administrator of a college was berating private industry. She said something to the effect of: “Businesses are terrible. There are a record number of open cyber positions, yet businesses aren’t hiring those applicants – our graduates – for those open positions, and as a result our alumni are unemployed.” I’m an employer of a variety of employees – including some who do NOT have a college degree – and have found that the degree is not meaningful in the field of applied cybersecurity research. I want problem solvers. Give me those and everybody wins.


The solution to the needs.

We believe there’s a single solution to address the opportunities posed by the above needs.

(1) The ideal candidates for Cyber Operations positions are those who are given enough access, continual attempts, and real chances to solve COAC-like puzzles. Find the person who loves and thrives on hard problems – and know that this is the person who should go to cyber ops academy.

(2) Applicants for a position at Point3 are given a handful of challenges and a limited time of engagement. We know you aren’t going to solve all of them in the allotted time. That’s not the point.  But, we can see your thought process, design decisions, and ability to show a thorough understanding of the problem set. Someone who doesn’t give up, who has breakthroughs and can progress efficiently – that’s the person we want (college degree or not).

(3) Clearly, there’s a disconnect between academia and industry; I personally do not believe that industry certs are the sole answer. When we then pile on ‘important definitions’ on top of ‘important concepts’ we still fail on anything applied; on anything lab-driven; and on building one’s own solution instead of copy/pasting the first Google search hit. What the market needs is an ecosystem that provides those who want to learn with a true outlet to learn and HR professionals the ability to assess before making potentially costly hiring decisions.


Presenting ESCALATE.

ESCALATE is the vehicle through which Point3 solves the needs of talent identification, validation, and cultivation. The secret sauce? Compelling content from experienced operations professionals. The challenges are based upon three years of COAC and a lifetime of operational experience.

Members of the ESCALATE ecosystem get access to the challenges,

access to the COAC instructors

and a slew of video and text based resources for help.

They don’t spend $5,000+ to bask in the presence of some luminary at a one-week boot camp. They become the luminary.

Skills take time and effort to develop.  Everyone wants the muscles but few invest the energy to lift the heavy thing up and down in repetition for months.  Of course a boot camp does not work!  You cram for a test and forget everything the next day; never retaining or applying a thing! Zero ROI for an employer.

ESCALATE is coming, and here’s why it matters: developing applied skills is not satiated in books or in Victorian style classrooms. Our industry is a craft. It’s time to return to tradecraft – one that matches master craftsmen with apprentices.

With ESCALATE we can track who’s logging in, when, and for how long. We can see progress and infer working through frustration. We can monitor for engagement and achievement. ESCALATE is the ASVAB for Cyber. It allows the ‘right’ talent to self-identify.

ESCALATE can be used to measure an individual against a specific position – we map each challenge to both the NICE and CAE frameworks. And it can also be used to measure the strengths (and weaknesses) of a team across a set of disciplines.

The hiring gap doesn’t exist because businesses are terrible. It exists because job seekers that can’t hit the ground running are too expensive to onboard. It exists because employers are sick of getting burned on wrong hires. It exists because the educational system is teaching concepts and failing to provide an authentic experience that industry values. Let’s change all this. Together.


Call to action.

ESCALATE will be available to the public on 27 November (Happy Cyber-Monday, everyone!).

The only place you can get it is on Cybrary. 

Pre-order before launch and we’ll toss in a copy of BinaryNinja (a $150 value).


[1] Certified Information Systems Security Professional:

[2] Certified Ethical Hacker:

[3] PS I think the answer is the ‘Reconnaissance’ phase, but I don’t know.  Perhaps it’s the “Pre-Attack” phase.   A professional operator might consider scanning part of the post-attack too, because that’s when one gets to scour a system for value.  But on CEH there is only one correct answer on the multiple-choice test, so you better have their answer memorized when you sit for the exam!

[4] CompTia Security+ Objective 3.1 – Explain types of malware.

[5] “I don’t believe in the No-Win Scenario.”








Share this post and earn Cybytes
Facebook Twitter LinkedIn Email
About Point3 Security, Inc.
Since our inception, Point3 Security, Inc. has continued to disrupt the cybersecurity and defense education industry by dismantling the traditional classroom education model, and developing a gamified, proven, challenge-based ecosystem. Escalate™, our challenge based ecosystem, introduces hands-on-keyboard capture-the-flag tasks which mentor, train, and teach members master tradecraft to malware reverse engineering, network security, disk and memory forensics, vulnerability research, offensive and defensive operations, and operational software development — skills sought by the U.S. Government, and equally rewarded in the private/commercial sector through immediate senior level salary, job security, and genuine job satisfaction. Unlike participants in expensive cyber boot-camp gimmicks, and generic cybersecurity certificate curriculums riddling the net, Escalate™ members continue to be identified as the gold-standard of the cybersecurity and defense industry by government agency recruiters, and commercial sector head-hunters. Our award-winning method is recognized. The success of our members is measured and proven. Our process is validated. Point3 Security Inc. didn’t just challenge the industry standard; it redefined it.

Our Revolution

We believe Cyber Security training should be free, for everyone, FOREVER. Everyone, everywhere, deserves the OPPORTUNITY to learn, begin and grow a career in this fascinating field. Therefore, Cybrary is a free community where people, companies and training come together to give everyone the ability to collaborate in an open source way that is revolutionizing the cyber security educational experience.

Cybrary On The Go

Get the Cybrary app for Android for online and offline viewing of our lessons.

Get it on Google Play

Support Cybrary

Donate Here to Get This Month's Donor Badge

Skip to toolbar

We recommend always using caution when following any link

Are you sure you want to continue?