AI engineer. Previously co-founder and CTO at WatchMouse. Building https://paperzilla.ai
![Sycophancy under misleading user suggestions
With regard to the experiments conducted to measure sycophancy, we designed three distinct tests, with the aim of
identifying which method of presenting the incorrect suggestion had the greatest influence on the models’ performance.
The intention is to quantify the effect of an incorrect instruction provided within the prompt on the final response to the
question.
Sycophancy has been widely discussed in the literature as a tendency for models to adopt an excessively deferential
attitude towards the user, aligning themselves with the user’s statements even when these are incorrect. This phenomenon
has also been recognised by model developers, who have taken steps to mitigate it, sometimes drawing criticism from
some users.
In our case, sycophancy was tested by introducing into the prompt, following the text of the problem, a section of the
type: “My take: In my opinion the right answer is [. . . ], because. . . ”. In this sense, the experiment measures more
precisely the effect of an incorrect suggestion on the model’s overall performance. The three variants considered differ
in the type of justification associated with the incorrect answer. In the first case, we provided only an incorrect numerical
answer, selected not at random but in such a way as to reflect the typical outcome of probabilistic cognitive biases that
would influence a human subject. In the second case, the same answer was accompanied by an ad hoc justification
containing errors and logical fallacies consistent with such biases. In the third case, we adopted a different approach:
for each exercise, we used as the incorrect justification an answer actually produced previously by one of the models
in the tests without sycophancy, selecting the one from the model with the best average performance among those
that had nevertheless made a mistake on that exercise. In other words, instead of artificially constructing a fallacious
justification, we reused an inc…](https://cdn.bsky.app/img/feed_thumbnail/plain/did:plc:gf4ewk42xvbpnrq3k2yapgqk/bafkreihz4fxc6kcvi6p5fn6mfgbzddh53v6cgjpw7tvhcfzfrm6l7l2jky)
![Schematic neuron models. Left: standard neuron model in ANNs, following a simplistic abstraction from the neuroscience of the 1950s [1, 2] that assumed linear dendrites and an output nonlinearity. Right: proposed neuron model, based on a recent model of cortical cells [22]; notice how this new model extends the standard neuron model by considering internal nonlinearities σ (that represent dendritic processes), and these nonlinearities are affected in a dynamic way by the activity value zi (that represents the bAP going from the soma to the dendrites), itself depending on the inputs xj and the contributions zk from other neurons.](https://cdn.bsky.app/img/feed_thumbnail/plain/did:plc:gf4ewk42xvbpnrq3k2yapgqk/bafkreiawibegmydidzlvesz7k7zyul4bfinhwf4vitenfxksvqfqsammdm)
![Overall architecture of AlphaFlowTSE. Given a mixture waveform y and an enrollment utterance e, we compute complex STFT features and form the mixture feature Y and enrollment feature E (real/imaginary concatenation). During training, the backbone takes the current state feature zt; during inference we initialize z0 = Y . The enrollment feature is concatenated as a temporal prefix, yielding [E∥zt] (or [E∥z0] at inference), which is fed to the UDiT backbone. The backbone is conditioned via AdaLN on the absolute time t and the interval length ∆ = r − t (with r = 1 at inference), and predicts the mean velocity for finite-interval transport, denoted uθ(t, r, [E∥zt]). One-step inference (NFE= 1) produces an estimated complex STFT Sˆ = (SˆRe, SˆIm), which is converted to the
target waveform sˆ by iSTFT. The dashed module is an optional mixing-ratio predictor used only in the background-to-target ablation to predict the start coordinate](https://cdn.bsky.app/img/feed_thumbnail/plain/did:plc:gf4ewk42xvbpnrq3k2yapgqk/bafkreifnplebiv5xo5cehvlwymtxfew2udqxz54nnx2odypp3wl4z7q7vm)
AI engineer. Previously co-founder and CTO at WatchMouse. Building https://paperzilla.ai