<\/em><\/p>\r\n\r\n
Let’s look at how we bring on new talent and optimize the process.<\/p>\r\n\r\n In the approach above, we don’t have a mechanism for technical filtering ahead of technical interviews. The process will struggle to scale.<\/p>\r\n\r\n Reducing the area under the curve allows us to scale our hiring practices.<\/p>\r\n A well-designed test<\/em> should:<\/p>\r\n They’re a great aid to introduce to any organization that has begun to significantly scale.<\/p>\r\n However, Copilot\/ChatGPT and other ML models have the ability to compose reasonable functions from written descriptions<\/strong>, meaning outdated tests are trivially answerable without the candidate performing any action.<\/p>\r\n 2025 brings us to the point of needing to update our testing process to retain the strengths of administering short tests and to ensure we’re picking up talent that can handle complex problems outside of the scope of a query to copilot.<\/p>\r\n For teams that haven’t run their code tests through Copilot, they may be surprised to learn how well it performs on many types of questions. Code generation is an active area of research, and specialized models are able to obtain satisfactory answers to written prompt questions as evaluated by a human on upwards of 80% of questions.<\/p>\r\n We’ll explore what makes some questions trival and what makes them challenging.<\/p>\r\n\r\n <\/p>\r\n<\/blockquote>\r\n\r\n\r\n\r\n\r\n\r\n Tests are suitable for all roles that involve technical skills. The content will vary and may not involve “code” for some positions. Good tests for candidates typically have a few qualities in common:<\/p>\r\n a) a quick test: no more than 1-2 hours in length<\/p>\r\n b) low burden: quickly understandable by hiring lead, reduce workload<\/p>\r\n c) valuable: the answers should tell us quite a bit about the candidate<\/p>\r\n\r\n Getting the right balance of question types helps improve the results we obtain from our pipeline. 1) an easy question with a straightforward answer<\/strong><\/p>\r\n – this is a fast-fail for folks who are obvious non-fits 2) a harder question that involves solving something with two distinct tasks<\/strong><\/p>\r\n – this question should confound attempts to google\/copilot an answer 3) a question that involves debugging a snippet of code and fixing the errors<\/strong><\/p>\r\n – debugging code or data is a critical skill, one that’s harder to evaluate with in-person interviews 4) offer commentary about the above code and how you would improve it \/ add to it<\/strong><\/p>\r\n – being able to communicate with other developers about code is a key skill 5) a final question with an optional level of difficulty<\/strong><\/p>\r\n – lets us spot candidates with a depth of skill <\/p>\r\n\r\n\r\n\r\n<\/div><\/div>\r\n\r\n Having a refined design like the Five Questions <\/em>Template above gives us something we can quickly scan<\/strong> candidates and filter out obvious non-fits on a performance basis<\/strong> rather than assumptive.<\/p>\r\n During any hiring cycle, we’d like a good return for developer\/leadership\/HR time spent and<\/em>\u00a0 have a broad funnel for potential candidates. Excessive commitments of candidate time narrows this field, so a quick test<\/strong> outperforms a long test in the real world.<\/p>\r\n Tests that build our confidence in a candidate<\/strong> accelerate the in-person process; we can shorten the phase of the interview that establishes basic skill and re-commit that energy to fit and aptitude and a more finely tuned evaluation of particularly valuable skills.<\/p>\r\n![\"\"](\"https:\/\/lampwrite.com\/j\/wp-content\/uploads\/2024\/11\/3stage_interview.drawio.png\")
<\/figure>\r\n\r\nImproving the curve through testing<\/h3>\r\n\r\n\r\n\r\n
![\"\"](\"https:\/\/lampwrite.com\/j\/wp-content\/uploads\/2024\/11\/4stage_interview.drawio.png\")
<\/figure>\r\n\r\n\r\n
Good quality Code Tests<\/em> are a top line improvement to any pipeline<\/h3>\r\n\r\n
\r\n
A Quick Review of Testing Process<\/h3>\r\n\r\n\r\n\r\n
Versions of the test can be produced for customer-facing developers, data analyst and data scientist roles, UI and UX positions. Generally, the overarching lessons here apply to all technical testing, not just development.<\/p>\r\n\r\n
Here’s a classic outline of the “Five Questions” test.
<\/p>\r\n
– being a straightforward question, it reduces test anxiety and gives us better data on potential post-hire performance
– it shouldn’t take more than 5 minutes to answer question<\/p>\r\n
– it should take 15-30 minutes to complete an answer<\/p>\r\n
– moving off hypothetical \/ clean room problems helps us explore capabilities
– the candidate being able to spot additional errors and side effects, even if they aren’t readily apparent, is a great to watch for
– it should take 25 minutes to 45 minutes to answer<\/p>\r\n
– knowing what the candidate values is important and quite informative
– knowing what the person regards as ‘problems’ and how they address them tells us about personality fit
– should take not more than 15 minutes to answer
– questions 4 & 5 benefit from not having a reduced time pressure<\/p>\r\n
– should have many deep areas to explore in greater detail in a technical interview
– all of the remaining time (25 to 60 minutes, plus more as desired)<\/p>\r\n