Search Options

Display Count
Sort
Preferred Language
Advanced Search

Results 1 - 10 of 43 for precision (0.26 seconds)

  1. migrator/migrator.go

    				alterColumn = true
    			}
    		}
    	} else {
    		if precision, _, ok := columnType.DecimalSize(); ok && int64(field.Precision) != precision {
    			if regexp.MustCompile(fmt.Sprintf("[^0-9]%d[^0-9]", field.Precision)).MatchString(m.DataTypeOf(field)) {
    				alterColumn = true
    			}
    		}
    	}
    
    	// check nullable
    	if nullable, ok := columnType.Nullable(); ok && nullable == field.NotNull {
    Created: Sun Apr 05 09:35:12 GMT 2026
    - Last Modified: Sat Mar 21 11:49:01 GMT 2026
    - 29.8K bytes
    - Click Count (0)
  2. android/guava-tests/test/com/google/common/math/LinearTransformationTest.java

        assertThat(transformation.slope()).isWithin(ALLOWED_ERROR).of(slope);
        // Note that we cannot test the actual mapping of points, as the results will be unreliable due
        // to loss of precision with this value of the slope.
      }
    
      public void testMappingWithSlope_maximalSlope() {
        double x1 = 1.2;
        double y1 = 3.4;
        double slope = Double.MAX_VALUE;
    Created: Fri Apr 03 12:43:13 GMT 2026
    - Last Modified: Tue Mar 17 16:11:48 GMT 2026
    - 7.2K bytes
    - Click Count (0)
  3. guava-tests/test/com/google/common/math/LinearTransformationTest.java

        assertThat(transformation.slope()).isWithin(ALLOWED_ERROR).of(slope);
        // Note that we cannot test the actual mapping of points, as the results will be unreliable due
        // to loss of precision with this value of the slope.
      }
    
      public void testMappingWithSlope_maximalSlope() {
        double x1 = 1.2;
        double y1 = 3.4;
        double slope = Double.MAX_VALUE;
    Created: Fri Apr 03 12:43:13 GMT 2026
    - Last Modified: Tue Mar 17 16:11:48 GMT 2026
    - 7.2K bytes
    - Click Count (0)
  4. android/guava/src/com/google/common/math/Quantiles.java

         * Computes the quantile value of the given dataset.
         *
         * @param dataset the dataset to do the calculation on, which must be non-empty, which will be
         *     cast to doubles (with any associated lost of precision), and which will not be mutated by
         *     this call (it is copied instead)
         * @return the quantile value
         */
        public double compute(Collection<? extends Number> dataset) {
    Created: Fri Apr 03 12:43:13 GMT 2026
    - Last Modified: Mon Mar 23 21:06:42 GMT 2026
    - 30.1K bytes
    - Click Count (0)
  5. src/cmd/asm/internal/asm/testdata/riscv64.s

    	FNMADDS	F1, F2, F3, F4				// 4f822018
    
    	// 20.8: Single-Precision Floating-Point Compare Instructions
    	FEQS	F0, F1, X7				// d3a300a0
    	FLTS	F0, F1, X7				// d39300a0
    	FLES	F0, F1, X7				// d38300a0
    
    	// 20.9: Single-Precision Floating-Point Classify Instruction
    	FCLASSS	F0, X5					// d31200e0
    
    	// 21.3: Double-Precision Load and Store Instructions
    	FLD	(X5), F0				// 07b00200
    	FLD	4(X5), F0				// 07b04200
    Created: Tue Apr 07 11:13:11 GMT 2026
    - Last Modified: Sat Apr 04 05:25:40 GMT 2026
    - 74.2K bytes
    - Click Count (0)
  6. tests/migrate_test.go

    			}
    		case "salary":
    			if precision, o, _ := columnType.DecimalSize(); (fullSupported || precision != 0) && precision != 2 {
    				t.Fatalf("salary's precision should be 2, but got %v %v", precision, o)
    			}
    		case "birthday":
    			if precision, _, _ := columnType.DecimalSize(); (fullSupported || precision != 0) && precision != 2 {
    				t.Fatalf("birthday's precision should be 2, but got %v", precision)
    			}
    		}
    	}
    
    Created: Sun Apr 05 09:35:12 GMT 2026
    - Last Modified: Sat Mar 21 11:49:01 GMT 2026
    - 66.3K bytes
    - Click Count (0)
  7. doc/go_spec.html

    to the precision specified by the destination type.
    For instance, the value of a variable <code>x</code> of type <code>float32</code>
    may be stored using additional precision beyond that of an IEEE 754 32-bit number,
    but float32(x) represents the result of rounding <code>x</code>'s value to
    32-bit precision. Similarly, <code>x + 0.1</code> may use more than 32 bits
    Created: Tue Apr 07 11:13:11 GMT 2026
    - Last Modified: Wed Apr 01 23:39:18 GMT 2026
    - 287.8K bytes
    - Click Count (1)
  8. RELEASE.md

    *   TF Core:
    
        *   Certain float32 ops run in lower precision on Ampere based GPUs,
            including matmuls and convolutions, due to the use of
            [TensorFloat-32](https://blogs.nvidia.com/blog/2020/05/14/tensorfloat-32-precision-format/).
            Specifically, inputs to such ops are rounded from 23 bits of precision
            to 10 bits of precision. This is unlikely to cause issues in practice
    Created: Tue Apr 07 12:39:13 GMT 2026
    - Last Modified: Mon Mar 30 18:31:38 GMT 2026
    - 746.5K bytes
    - Click Count (3)
  9. android/guava/src/com/google/common/primitives/Floats.java

          return Arrays.copyOfRange(array, start, end);
        }
    
        @GwtIncompatible @J2ktIncompatible private static final long serialVersionUID = 0;
      }
    
      /**
       * Parses the specified string as a single-precision floating point value. The ASCII character
       * {@code '-'} (<code>'&#92;u002D'</code>) is recognized as the minus sign.
       *
       * <p>Unlike {@link Float#parseFloat(String)}, this method returns {@code null} instead of
    Created: Fri Apr 03 12:43:13 GMT 2026
    - Last Modified: Mon Mar 23 16:38:16 GMT 2026
    - 25.6K bytes
    - Click Count (0)
  10. android/guava/src/com/google/common/primitives/Doubles.java

                "#",
                "+"
                );
        return
        java.util.regex.Pattern
            .compile(fpPattern);
      }
    
      /**
       * Parses the specified string as a double-precision floating point value. The ASCII character
       * {@code '-'} (<code>'&#92;u002D'</code>) is recognized as the minus sign.
       *
       * <p>Unlike {@link Double#parseDouble(String)}, this method returns {@code null} instead of
    Created: Fri Apr 03 12:43:13 GMT 2026
    - Last Modified: Mon Mar 23 16:38:16 GMT 2026
    - 27.8K bytes
    - Click Count (0)
Back to Top