Xjyuk

In most textbooks, dot product between two vectors is defined as:

$$langle x_1,x_2,x_3rangle cdot langle y_1,y_2,y_3rangle = x_1 y_1 + x_2 y_2 + x_3 y _3$$

I understand how this definition works most of the time. However, in this definition, there is no reference to coordinate system (i.e. no basis is included for the vector components). So, if I had two vectors in two different coordinate systems:

$$x_1 vece_1 + x_2 vece_2 + x_3 vece_3$$
$$y_1 vece_1' + y_2 vece_2' + y_3 vece_3'$$

How, would I compute their dot product? In particular, is there a more formal/abstract/generalized definition of the dot product (that would allow me to compute $vece_1 cdot vece_1'$ without converting the vectors to the same coordinate system)? Even if I did convert the vectors to the same coordinate system, why do we know that the result will be the same if I multiply the components in the primed system versus in the unprimed system?

Your top-line question can be answered at many levels. Setting aside issues of forms and covariant/contravariant, the answer is:

The dot product is the product of the magnitudes of the two vectors, times the cosine of the angle between them.

No matter what basis you compute that in, you have to get the same answer because it's a physical quantity.

The usual "sum of products of orthonormal components" is then a convenient computational approach, but as you've seen it's not the only way to compute them.

The dot product's properties includes linear, commutative, distributive, etc. So when you expand the dot product

$$(a_x hatx+a_y haty + a_z hatz) cdot (b_x hatX+b_y hatY + b_z hatZ)$$

you get nine terms like $( a_x b_x hatxcdothatX) + (a_x b_y hatxcdothatY)+$ etc. In the usual orthonormal basis, the same-axis $hatxcdothatX$ factors just become 1, while the different-axis $hatxcdothatY$ et al factors are zero. That reduces to the formula you know.

In a non-orthonormal basis, you have to figure out what those basis products are. To do that, you refer back to the definition: The product of the size of each, times the cosine of the angle between. Once you have all of those, you're again all set to compute. It just looks a bit more complicated...

The dot product can be defined in a coordinate-independent way as

$$vecacdotvecb=|veca||vecb|costheta$$

where $theta$ is the angle between the two vectors. This involves only lengths and angles, not coordinates.

To use your first formula, the coordinates must be in the same basis.

You can convert between bases using a rotation matrix, and the fact that a rotation matrix preserves vector lengths is sufficient to show that it preserves the dot product. This is because

$$vecacdotvecb=frac12left(|veca+vecb|^2-|veca|^2-|vecb|^2right).$$

This formula is another purely-geometric, coordinate-free definition of the dot product.

On computing the following matrix will give you the dot product $$beginbmatrix x_1 & x_2& x_3 endbmatrix.beginbmatrix e_1.e'_1 & e_1.e'_2 & e_1.e'_3 \ e_2.e'_1 & e_2.e'_2 & e_2.e'_3 \ e_3.e'_1 & e_3.e'_2 & e_3.e_3endbmatrix.beginbmatrixy_1\y_2\y_3endbmatrix$$ If we transform the cordinate of the a vector, only the components and basis of vector changes. The vector remains unchanged. Thus the dot product remain unchanged even if we compute dot product between primed and unprimed vectors.

The coordinate free definition of a dot product is:

$$ vec a cdot vec b = frac 1 4 [(vec a + vec b)^2 - (vec a - vec b)^2]$$

It's up to you to figure out what the norm is:

$$ ||vec a|| = sqrt(vec a)^2$$