GPT-4 and professional benchmarks, the wrong answer to the wrong question, How V8 JavaScript engine works step by step [with diagram], Unclogging the Bug Pipeline, CSS property order
As further evidence for this hypothesis, we tested it on Codeforces problems from different times in 2021. We found that it could regularly solve problems in the easy category before September 5, but none of the problems after September 12.
In fact, we can definitively show that it has memorized problems in its training set: when prompted with the title of a Codeforces problem, GPT-4 includes a link to the exact contest where the problem appears (and the round number is almost correct: it is off by one). Note that GPT-4 cannot access the Internet, so memorization is the only explanation.
[…]
Because of OpenAI’s lack of transparency, we can’t answer the contamination question with certainty. But what’s certain is that OpenAI’s method to detect contamination is superficial and sloppy
[…]
On top of this, professional exams, especially the bar exam, notoriously overemphasize subject-matter knowledge and underemphasize real-world skills, which are far harder to measure in a standardized, computer-administered way. In other words, not only do these exams emphasize the wrong thing, they overemphasize precisely the thing that language models are good at.
Un regard critique sur les études de performance des versions de GPT et autres modèles.
From a high-level view, the V8 JavaScript engine execution consists of 5 steps.
- Initialize environment in the host
- Compile JavaScript codes
- Generate bytecodes
- Interpret and execute bytecodes
- Optimize some bytecodes for better performance
V8, moteur d'exécution de Javascript au sein de Chromium mais aussi de NodeJS, under the hood comme on dit. Notez les deux pointeurs eux aussi intéressants "how the call stack works" et "the event loop"
Let’s call that the “kinetic” model of the test process. There is truth to that model, because we can ask “why don’t the bugs just fall right out when we want them?” The answers to that are interesting and deep. The product, the tester, and the test process itself— all these things together— can be thought of as a sieve with different filters that can block the good bugs from being reported. Part of testing well is to keep those filters unclogged. But they are not just filters, they are also steps in a chain of evidence. When you report a bug you are reporting an unbroken chain of reasoning that spans “this is a problem that matters” back to “this exists.”
[…]
Since it is both a sieve and a chain, let’s call it something that evokes both ideas: a bug reporting pipeline. Let’s look closely at it and how it can go wrong.
Un article top d'un acolyte de Michael Bolton par lequel j'ai beaucoup appris sur la complexité humaine, technique et organisationnelle que rencontrent les testeurs. Je pense que c’est autant utile de comprendre ce qu’il se passe durant cette phase pour mieux préparer les choses en dev et design.
CSS property order matters, especially on larger teams with several engineers and designers all contributing code. I know it’s been discussed, but frankly there’s always room for improvement and alternate approaches.
Testé et approuvé. On a utilisé cette approche sur quelques missions où nous n'avions pas de framework qui change fondamentalement la façon d'écrire du CSS (spécifiquement PostCSS, permettant d'écrire du CSS3 avec quelques améliorations qui transpile pour rendre compatible avec tout type de navigateur). C'est d'autant plus jouable en utilisant un linter de css qui permet de faire du fix à notre place comme stylelint.
C'est d'ailleurs ce qu'utilise ce blog : PostCSS avec une poignée de plugins(la conf a légèrement évolué depuis), et le linter stylelint avec le plugin stylelint-config-recess-order. J'utilise notamment la configuration avancée dans cette page pour avoir un retour à la ligne entre les sections.
L'avantage c'est que lorsqu'on tombe sur CSS qu'on a pas écrit, la structure et l'ordre des propriétés étant strictement les mêmes sur toute la base de code, on s'y retrouve très facilement. Au point où, même sur des projets au style très sur mesure, le style est absolument maitrisable sans framework.
