Mesa (glsl2): glsl2: Fix expected type for multiplying vector with non-square matrix.

[Carl Worth]

Module: Mesa
Branch: glsl2
Commit: 47c90b144729e3edf3b5cbf5b260c1c46e429879
URL:    http://cgit.freedesktop.org/mesa/mesa/commit/?id=47c90b144729e3edf3b5cbf5b260c1c46e429879

Author: Carl Worth <cworth at cworth.org>
Date:   Thu Jul 22 14:56:14 2010 -0700

glsl2: Fix expected type for multiplying vector with non-square matrix.

Previously, the compiler expected the result of the multiplication to
be of the same type as the vector. This is correct for square
matrices, but wrong for all others.

We fix this by instead expecting a vector with the same number of rows
as the matrix (for the case of M*v with a column vector) or the same
number of columns as the matrix (for v*M with a row vector).

This fix causes the following four glean tests to now pass:

	glsl1-mat4x2 * vec4
  	glsl1-vec2 * mat4x2 multiply
  	glsl1-vec3 * mat4x3 multiply
  	glsl1-vec4 * mat3x4 multiply

---

 src/glsl/ast_to_hir.cpp |   26 ++++++++++++++++++++++----
 1 files changed, 22 insertions(+), 4 deletions(-)

diff --git a/src/glsl/ast_to_hir.cpp b/src/glsl/ast_to_hir.cpp
index 0cb3863..5e26f21 100644
--- a/src/glsl/ast_to_hir.cpp
+++ b/src/glsl/ast_to_hir.cpp
@@ -282,8 +282,17 @@ arithmetic_result_type(ir_rvalue * &value_a, ir_rvalue * &value_b,
 	  * means the vector type of a row from A must be the same as the
 	  * vector the type of B.
 	  */
-	 if (type_a->row_type() == type_b)
-	    return type_b;
+	 if (type_a->row_type() == type_b) {
+	    /* The resulting vector has a number of elements equal to
+	     * the number of rows of matrix A. */
+	    const glsl_type *const type =
+	       glsl_type::get_instance(type_a->base_type,
+				       type_a->column_type()->vector_elements,
+				       1);
+	    assert(type != glsl_type::error_type);
+
+	    return type;
+	 }
       } else {
 	 assert(type_b->is_matrix());
 
@@ -292,8 +301,17 @@ arithmetic_result_type(ir_rvalue * &value_a, ir_rvalue * &value_b,
 	  * the type of A must be the same as the vector type of a column from
 	  * B.
 	  */
-	 if (type_a == type_b->column_type())
-	    return type_a;
+	 if (type_a == type_b->column_type()) {
+	    /* The resulting vector has a number of elements equal to
+	     * the number of columns of matrix B. */
+	    const glsl_type *const type =
+	       glsl_type::get_instance(type_a->base_type,
+				       type_b->row_type()->vector_elements,
+				       1);
+	    assert(type != glsl_type::error_type);
+
+	    return type;
+	 }
       }
 
       _mesa_glsl_error(loc, state, "size mismatch for matrix multiplication");