Xjyuk

In the documentation of l3skip's dim_eval:n one is warned

This [...] requires suitable termination if used in a TeX-style assignment as it is not an ⟨internal dimension⟩.

Analogous warnings can be found in the descriptions for skip_eval:n and muskip_eval:n with ⟨internal glue⟩ and ⟨internal muglue⟩ in place of ⟨internal dimension⟩.

I am fairly familiar with handling skips and dimensions in TeX, but I have never before come across the expression internal dimension. In the TeXbook I could not find any explanation of the phrase outside of the rather technical grammar notation in chapter 24, which surely contains all the necessary information but did not really help me.

So, what exactly is an internal dimension? What are other types of dimensions? (External dimensions?) What are examples of different behavior of the two (like the one alluded to by the warning above)?

I'll deal first with the general TeX concept, then why it's important in the documentation of those expl3 functions.

An internal dimension (or internal count or whatever) is something which has been parsed by TeX and is now stored in the correct form. Thus TeX 'knows' that an internal dimension is a valid dimension, and does not have to 'look' for any further material. In contrast, an external dimension (etc.) is something that is made up of discrete tokens and would have to be re-parsed by TeX to be used. Thus when we write 12.0pt, we are giving an external representation (TeX would have to parse it to know it's a valid dimen), but after

newdimenmydimen
mydimen=12pt %

I can use mydimen and TeX does not need to parse anything: mydimen holds an internal dimension.

Why is this important? It's all about TeX's parsing rules, in particular that TeX allows an optional trailing space after dimensions, integers, etc., and more importantly that with an external representation, TeX doesn't stop parsing until it finds something that doesn't 'fit'. For example

deffoo123
newcountfooint
fooint=123 %
newcounttestint
%%%
testint=foo 456 %
showthetestint
testint=fooint 456 %
showthetestint

you'll see that the first case gives the wrong result: we have a macro which simply expands to 123, and TeX keeps looking for an integer until we hit the optional space. In contrast, with an internal count representation, there is no question of parsing: fooint is 123.

The key point is that an internal representation is 'safer' to use (plus faster): there's never a question of where it terminates.

How does this relate to expl3? Something like dim_eval:n is used to take an expression and turn in into a dimension. However, it turns out to be convenient to allow that to also be just typeset, stored by expansion in a macro (tl), etc. To do that, we have to arrange that the evaluation results in an external representation, not an internal one. That means that these functions behave like storing a value as a macro: you have to watch the termination.

For all 'pure' expl3 usage that's not an issue, as we have correct termination in the right places. But if you mix using these functions with more classical TeX programming, you need to know how they will behave. The answer by egreg shows this nicely.

For those who want the TeX details, dim_eval:n is in primitive terms

thedimexpr #1relax

while if we want to end up with an internal representation we only want

dimexpr #1relax

However, that can't be used in typesetting or (successfully) inside an x-type expansion, hence it's not suitable for the definition we want.

Consider the following example

documentclassarticle
usepackageexpl3

ExplSyntaxOn
cs_set_eq:NN dimeval dim_eval:n
ExplSyntaxOff

newlengthmylen

begindocument

Do an assignment mylen=dimeval3pt+1cm plus something else.

enddocument

which raises an error

! Missing number, treated as zero.
<to be read again> 
 s
l.12 ...assignment mylen=dimeval3pt+1cm plus s
 omething else.

This is precisely what's referred to in interface3.

An <internal dimension> is any dimen register or any internal register that stores a (rigid) length, such as parindent; with e-TeX extensions, also dimexpr is an instance of <internal dimension>.

Another important fact is that newlength allocates a skip register, so TeX will look forward for plus or minus specifications; this doesn't occur when setlength is used, because the macro provides the suitable relax termination.