# NumPy 1.21.0 Release Notes

The NumPy 1.21.0 release highlights are

- continued SIMD work covering more functions and platforms,

- initial work on the new dtype infrastructure and casting,

- universal2 wheels for Python 3.8 and Python 3.9 on Mac,

- improved documentation,

- improved annotations,

- new `PCG64DXSM` bitgenerator for random numbers.

In addition there are the usual large number of bug fixes and other improvements.

The Python versions supported for this release are 3.7-3.9. Official support for Python 3.10 will be added when it is released.

Warning

There are unresolved problems compiling NumPy 1.20.0 with gcc-11.1.

- Optimization level -O3 results in many incorrect warnings when running the tests.

- On some hardware NumPY will hang in an infinite loop.

## New functions

### Add `PCG64DXSM` `BitGenerator`

Uses of the `PCG64` `BitGenerator` in a massively-parallel context have been shown to have statistical weaknesses that were not apparent at the first release in numpy 1.17. Most users will never observe this weakness and are safe to continue to use `PCG64`. We have introduced a new `PCG64DXSM` `BitGenerator` that will eventually become the new default `BitGenerator` implementation used by `default_rng` in future releases. `PCG64DXSM` solves the statistical weakness while preserving the performance and the features of `PCG64`.

See Upgrading PCG64 with PCG64DXSM for more details.

(gh-18906)

## Expired deprecations

- The `shape` argument `unravel_index` cannot be passed as `dims` keyword argument anymore. (Was deprecated in NumPy 1.16.)

  (gh-17900)

- The function `PyUFunc_GenericFunction` has been disabled. It was deprecated in NumPy 1.19. Users should call the ufunc directly using the Python API.

  (gh-18697)

- The function `PyUFunc_SetUsesArraysAsData` has been disabled. It was deprecated in NumPy 1.19.

  (gh-18697)

- The class `PolyBase` has been removed (deprecated in numpy 1.9.0). Please use the abstract `ABCPolyBase` class instead.

  (gh-18963)

- The unused `PolyError` and `PolyDomainError` exceptions are removed.

  (gh-18963)

## Deprecations

### The `.dtype` attribute must return a `dtype`

A `DeprecationWarning` is now given if the `.dtype` attribute of an object passed into `np.dtype` or as a `dtype=obj` argument is not a dtype. NumPy will stop attempting to recursively coerce the result of `.dtype`.

(gh-13578)

### Inexact matches for `numpy.convolve` and `numpy.correlate` are deprecated

`convolve` and `correlate` now emit a warning when there are case insensitive and/or inexact matches found for `mode` argument in the functions. Pass full `"same"`, `"valid"`, `"full"` strings instead of `"s"`, `"v"`, `"f"` for the `mode` argument.

(gh-17492)

### `np.typeDict` has been formally deprecated

`np.typeDict` is a deprecated alias for `np.sctypeDict` and has been so for over 14 years (6689502). A deprecation warning will now be issued whenever getting `np.typeDict`.

(gh-17586)

### Exceptions will be raised during array-like creation

When an object raised an exception during access of the special attributes `__array__` or `__array_interface__`, this exception was usually ignored. A warning is now given when the exception is anything but AttributeError. To silence the warning, the type raising the exception has to be adapted to raise an `AttributeError`.

(gh-19001)

### Four `ndarray.ctypes` methods have been deprecated

Four methods of the `ndarray.ctypes` object have been deprecated, as they are (undocumentated) implementation artifacts of their respective properties.

The methods in question are:

- `_ctypes.get_data` (use `_ctypes.data` instead)

- `_ctypes.get_shape` (use `_ctypes.shape` instead)

- `_ctypes.get_strides` (use `_ctypes.strides` instead)

- `_ctypes.get_as_parameter` (use `_ctypes._as_parameter_` instead)

(gh-19031)

## Expired deprecations

- The `shape` argument `numpy.unravel_index` cannot be passed as `dims` keyword argument anymore. (Was deprecated in NumPy 1.16.)

  (gh-17900)

- The function `PyUFunc_GenericFunction` has been disabled. It was deprecated in NumPy 1.19. Users should call the ufunc directly using the Python API.

  (gh-18697)

- The function `PyUFunc_SetUsesArraysAsData` has been disabled. It was deprecated in NumPy 1.19.

  (gh-18697)

### Remove deprecated `PolyBase` and unused `PolyError` and `PolyDomainError`

The class `PolyBase` has been removed (deprecated in numpy 1.9.0). Please use the abstract `ABCPolyBase` class instead.

Furthermore, the unused `PolyError` and `PolyDomainError` exceptions are removed from the `numpy.polynomial`.

(gh-18963)

## Compatibility notes

### Error type changes in universal functions

The universal functions may now raise different errors on invalid input in some cases. The main changes should be that a `RuntimeError` was replaced with a more fitting `TypeError`. When multiple errors were present in the same call, NumPy may now raise a different one.

(gh-15271)

### `__array_ufunc__` argument validation

NumPy will now partially validate arguments before calling `__array_ufunc__`. Previously, it was possible to pass on invalid arguments (such as a non-existing keyword argument) when dispatch was known to occur.

(gh-15271)

### `__array_ufunc__` and additional positional arguments

Previously, all positionally passed arguments were checked for `__array_ufunc__` support. In the case of `reduce`, `accumulate`, and `reduceat` all arguments may be passed by position. This means that when they were passed by position, they could previously have been asked to handle the ufunc call via `__array_ufunc__`. Since this depended on the way the arguments were passed (by position or by keyword), NumPy will now only dispatch on the input and output array. For example, NumPy will never dispatch on the `where` array in a reduction such as `np.add.reduce`.

(gh-15271)

### Validate input values in `Generator.uniform`

Checked that `high - low >= 0` in `np.random.Generator.uniform`. Raises `ValueError` if `low > high`. Previously out-of-order inputs were accepted and silently swapped, so that if `low > high`, the value generated was `high + (low - high) * random()`.

(gh-17921)

### `/usr/include` removed from default include paths

The default include paths when building a package with `numpy.distutils` no longer include `/usr/include`. This path is normally added by the compiler, and hardcoding it can be problematic. In case this causes a problem, please open an issue. A workaround is documented in PR 18658.

(gh-18658)

### Changes to comparisons with `dtype=...`

When the `dtype=` (or `signature`) arguments to comparison ufuncs (`equal`, `less`, etc.) is used, this will denote the desired output dtype in the future. This means that:

> np.equal(2, 3, dtype=object)

will give a `FutureWarning` that it will return an `object` array in the future, which currently happens for:

> np.equal(None, None, dtype=object)

due to the fact that `np.array(None)` is already an object array. (This also happens for some other dtypes.)

Since comparisons normally only return boolean arrays, providing any other dtype will always raise an error in the future and give a `DeprecationWarning` now.

(gh-18718)

### Changes to `dtype` and `signature` arguments in ufuncs

The universal function arguments `dtype` and `signature` which are also valid for reduction such as `np.add.reduce` (which is the implementation for `np.sum`) will now issue a warning when the `dtype` provided is not a “basic” dtype.

NumPy almost always ignored metadata, byteorder or time units on these inputs. NumPy will now always ignore it and raise an error if byteorder or time unit changed. The following are the most important examples of changes which will give the error. In some cases previously the information stored was not ignored, in all of these an error is now raised:

    # Previously ignored the byte-order (affect if non-native)
    np.add(3, 5, dtype=">i32")

    # The biggest impact is for timedelta or datetimes:
    arr = np.arange(10, dtype="m8[s]")
    # The examples always ignored the time unit "ns":
    np.add(arr, arr, dtype="m8[ns]")
    np.maximum.reduce(arr, dtype="m8[ns]")

    # The following previously did use "ns" (as opposed to `arr.dtype`)
    np.add(3, 5, dtype="m8[ns]")  # Now return generic time units
    np.maximum(arr, arr, dtype="m8[ns]")  # Now returns "s" (from `arr`)

The same applies for functions like `np.sum` which use these internally. This change is necessary to achieve consistent handling within NumPy.

If you run into these, in most cases pass for example `dtype=np.timedelta64` which clearly denotes a general `timedelta64` without any unit or byte-order defined. If you need to specify the output dtype precisely, you may do so by either casting the inputs or providing an output array using out=.

NumPy may choose to allow providing an exact output `dtype` here in the future, which would be preceded by a `FutureWarning`.

(gh-18718)

### Ufunc `signature=...` and `dtype=` generalization and `casting`

The behaviour for `np.ufunc(1.0, 1.0, signature=...)` or `np.ufunc(1.0, 1.0, dtype=...)` can now yield different loops in 1.21 compared to 1.20 because of changes in promotion. When `signature` was previously used, the casting check on inputs was relaxed, which could lead to downcasting inputs unsafely especially if combined with `casting="unsafe"`.

Casting is now guaranteed to be safe. If a signature is only partially provided, for example using `signature=("float64", None, None)`, this could lead to no loop being found (an error). In that case, it is necessary to provide the complete signature to enforce casting the inputs. If `dtype="float64"` is used or only outputs are set (e.g. `signature=(None, None, "float64")` the is unchanged. We expect that very few users are affected by this change.

Further, the meaning of `dtype="float64"` has been slightly modified and now strictly enforces only the correct output (and not input) DTypes. This means it is now always equivalent to:

    signature=(None, None, "float64")

(If the ufunc has two inputs and one output). Since this could lead to no loop being found in some cases, NumPy will normally also search for the loop:

    signature=("float64", "float64", "float64")

if the first search failed. In the future, this behaviour may be customized to achieve the expected results for more complex ufuncs. (For some universal functions such as `np.ldexp` inputs can have different DTypes.)

(gh-18880)

### Distutils forces strict floating point model on clang

NumPy distutils will now always add the `-ffp-exception-behavior=strict` compiler flag when compiling with clang. Clang defaults to a non-strict version, which allows the compiler to generate code that does not set floating point warnings/errors correctly.

(gh-19049)

## C API changes

### Use of `ufunc->type_resolver` and “type tuple”

NumPy now normalizes the “type tuple” argument to the type resolver functions before calling it. Note that in the use of this type resolver is legacy behaviour and NumPy will not do so when possible. Calling `ufunc->type_resolver` or `PyUFunc_DefaultTypeResolver` is strongly discouraged and will now enforce a normalized type tuple if done. Note that this does not affect providing a type resolver, which is expected to keep working in most circumstances. If you have an unexpected use-case for calling the type resolver, please inform the NumPy developers so that a solution can be found.

(gh-18718)

## New Features

### Added a mypy plugin for handling platform-specific `numpy.number` precisions

A mypy plugin is now available for automatically assigning the (platform-dependent) precisions of certain `number` subclasses, including the likes of `int_`, `intp` and `longlong`. See the documentation on scalar types for a comprehensive overview of the affected classes.

Note that while usage of the plugin is completely optional, without it the precision of above-mentioned classes will be inferred as `Any`.

To enable the plugin, one must add it to their mypy configuration file:

    [mypy]
    plugins = numpy.typing.mypy_plugin

(gh-17843)

### Let the mypy plugin manage extended-precision `numpy.number` subclasses

The mypy plugin, introduced in numpy/numpy#17843, has been expanded: the plugin now removes annotations for platform-specific extended-precision types that are not available to the platform in question. For example, it will remove `float128` when not available.

Without the plugin all extended-precision types will, as far as mypy is concerned, be available on all platforms.

To enable the plugin, one must add it to their mypy configuration file:

    [mypy]
    plugins = numpy.typing.mypy_plugin

(gh-18322)

### New `min_digits` argument for printing float values

A new `min_digits` argument has been added to the dragon4 float printing functions `format_float_positional` and `format_float_scientific` . This kwd guarantees that at least the given number of digits will be printed when printing in unique=True mode, even if the extra digits are unnecessary to uniquely specify the value. It is the counterpart to the precision argument which sets the maximum number of digits to be printed. When unique=False in fixed precision mode, it has no effect and the precision argument fixes the number of digits.

(gh-18629)

### f2py now recognizes Fortran abstract interface blocks

`f2py` can now parse abstract interface blocks.

(gh-18695)

### BLAS and LAPACK configuration via environment variables

Autodetection of installed BLAS and LAPACK libraries can be bypassed by using the `NPY_BLAS_LIBS` and `NPY_LAPACK_LIBS` environment variables. Instead, the link flags in these environment variables will be used directly, and the language is assumed to be F77. This is especially useful in automated builds where the BLAS and LAPACK that are installed are known exactly. A use case is replacing the actual implementation at runtime via stub library links.

If `NPY_CBLAS_LIBS` is set (optional in addition to `NPY_BLAS_LIBS`), this will be used as well, by defining `HAVE_CBLAS` and appending the environment variable content to the link flags.

(gh-18737)

### A runtime-subcriptable alias has been added for `ndarray`

`numpy.typing.NDArray` has been added, a runtime-subscriptable alias for `np.ndarray[Any, np.dtype[~Scalar]]`. The new type alias can be used for annotating arrays with a given dtype and unspecified shape. 1

1 NumPy does not support the annotating of array shapes as of 1.21, this is expected to change in the future though (see PEP 646).

#### Examples

    >>> import numpy as np
    >>> import numpy.typing as npt

    >>> print(npt.NDArray)
    numpy.ndarray[typing.Any, numpy.dtype[~ScalarType]]

    >>> print(npt.NDArray[np.float64])
    numpy.ndarray[typing.Any, numpy.dtype[numpy.float64]]

    >>> NDArrayInt = npt.NDArray[np.int_]
    >>> a: NDArrayInt = np.arange(10)

    >>> def func(a: npt.ArrayLike) -> npt.NDArray[Any]:
    ...     return np.array(a)

(gh-18935)

## Improvements

### Arbitrary `period` option for `numpy.unwrap`

The size of the interval over which phases are unwrapped is no longer restricted to `2 * pi`. This is especially useful for unwrapping degrees, but can also be used for other intervals.

    >>> phase_deg = np.mod(np.linspace(0,720,19), 360) - 180
    >>> phase_deg
    array([-180., -140., -100.,  -60.,  -20.,   20.,   60.,  100.,  140.,
           -180., -140., -100.,  -60.,  -20.,   20.,   60.,  100.,  140.,
           -180.])

    >>> unwrap(phase_deg, period=360)
    array([-180., -140., -100.,  -60.,  -20.,   20.,   60.,  100.,  140.,
            180.,  220.,  260.,  300.,  340.,  380.,  420.,  460.,  500.,
            540.])

(gh-16987)

### `np.unique` now returns single `NaN`

When `np.unique` operated on an array with multiple `NaN` entries, its return included a `NaN` for each entry that was `NaN` in the original array. This is now improved such that the returned array contains just one `NaN` as the last element.

Also for complex arrays all `NaN` values are considered equivalent (no matter whether the `NaN` is in the real or imaginary part). As the representant for the returned array the smallest one in the lexicographical order is chosen - see `np.sort` for how the lexicographical order is defined for complex arrays.

(gh-18070)

### `Generator.rayleigh` and `Generator.geometric` performance improved

The performance of Rayleigh and geometric random variate generation in `Generator` has improved. These are both transformation of exponential random variables and the slow log-based inverse cdf transformation has been replaced with the Ziggurat-based exponential variate generator.

This change breaks the stream of variates generated when variates from either of these distributions are produced.

(gh-18666)

### Placeholder annotations have been improved

All placeholder annotations, that were previously annotated as `typing.Any`, have been improved. Where appropriate they have been replaced with explicit function definitions, classes or other miscellaneous objects.

(gh-18934)

## Performance improvements

### Improved performance in integer division of NumPy arrays

Integer division of NumPy arrays now uses libdivide when the divisor is a constant. With the usage of libdivide and other minor optimizations, there is a large speedup. The `//` operator and `np.floor_divide` makes use of the new changes.

(gh-17727)

### Improve performance of `np.save` and `np.load` for small arrays

`np.save` is now a lot faster for small arrays.

`np.load` is also faster for small arrays, but only when serializing with a version >= `(3, 0)`.

Both are done by removing checks that are only relevant for Python 2, while still maintaining compatibility with arrays which might have been created by Python 2.

(gh-18657)

## Changes

### `numpy.piecewise` output class now matches the input class

When `ndarray` subclasses are used on input to `piecewise`, they are passed on to the functions. The output will now be of the same subclass as well.

(gh-18110)

### Enable Accelerate Framework

With the release of macOS 11.3, several different issues that numpy was encountering when using Accelerate Framework’s implementation of BLAS and LAPACK should be resolved. This change enables the Accelerate Framework as an option on macOS. If additional issues are found, please file a bug report against Accelerate using the developer feedback assistant tool (https://developer.apple.com/bug-reporting/). We intend to address issues promptly and plan to continue supporting and updating our BLAS and LAPACK libraries.

(gh-18874)