Skip to content

Dense

torchlinops.linops.Dense

Bases: NamedLinop

Dense matrix-vector multiply.

"Dense" is used to distinguish from "sparse" linear operators. This operator performs a matrix-vector multiplication, potentially with batch and broadcast dimensions, implemented via einops.einsum.

The core operation is:

\(y_{o\dots} = \sum_{i\dots} W_{i\dots, o\dots} x_{i\dots}\)

where \(x\) is the input, \(W\) is the weight matrix, and \(y\) is the output. \(i\dots\) and \(o\dots\) represent the input and output dimensions involved in the multiplication. Other dimensions are treated as batch or broadcast dimensions.

Examples:

A simple batched multiplication:

  • Input \(x\) shape: \((A, N_x, N_y)\)
  • Weight \(W\) shape: \((A, T)\)
  • Output \(y\) shape: \((T, N_x, N_y)\)

Here, \(A\) is the input feature dimension, \(T\) is the output feature dimension, and \((N_x, N_y)\) are broadcast dimensions. The operation is:

\(y_{t, n_x, n_y} = \sum_{a} W_{a, t} x_{a, n_x, n_y}\)

Another example with a batch dimension \(C\) shared between input and weights:

  • Input \(x\) shape: \((C, A, N_x, N_y)\)
  • Weight \(W\) shape: \((C, A, A_1)\)
  • Output \(y\) shape: \((C, A_1, N_x, N_y)\)

The operation is:

\(y_{c, a_1, n_x, n_y} = \sum_{a} W_{c, a, a_1} x_{c, a, n_x, n_y}\)

Source code in src/torchlinops/linops/dense.py
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
class Dense(NamedLinop):
    r"""Dense matrix-vector multiply.

    "Dense" is used to distinguish from "sparse" linear operators. This
    operator performs a matrix-vector multiplication, potentially with batch
    and broadcast dimensions, implemented via ``einops.einsum``.

    The core operation is:

    $y_{o\dots} = \sum_{i\dots} W_{i\dots, o\dots} x_{i\dots}$

    where $x$ is the input, $W$ is the weight matrix, and
    $y$ is the output. $i\dots$ and $o\dots$ represent
    the input and output dimensions involved in the multiplication. Other
    dimensions are treated as batch or broadcast dimensions.

    Examples
    --------
    A simple batched multiplication:

    - Input $x$ shape: $(A, N_x, N_y)$
    - Weight $W$ shape: $(A, T)$
    - Output $y$ shape: $(T, N_x, N_y)$

    Here, $A$ is the input feature dimension, $T$ is the output
    feature dimension, and $(N_x, N_y)$ are broadcast dimensions.
    The operation is:

    $y_{t, n_x, n_y} = \sum_{a} W_{a, t} x_{a, n_x, n_y}$

    Another example with a batch dimension $C$ shared between input
    and weights:

    - Input $x$ shape: $(C, A, N_x, N_y)$
    - Weight $W$ shape: $(C, A, A_1)$
    - Output $y$ shape: $(C, A_1, N_x, N_y)$

    The operation is:

    $y_{c, a_1, n_x, n_y} = \sum_{a} W_{c, a, a_1} x_{c, a, n_x, n_y}$

    """

    def __init__(
        self,
        weight: Tensor,
        weightshape: Shape | None = None,
        ishape: Shape | None = None,
        oshape: Shape | None = None,
        broadcast_dims: list | None = None,
    ):
        """
        Parameters
        ----------
        weight : Tensor
            The dense matrix used for this linop.
        weightshape : Shape, optional
            The shape of the matrix, in symbolic form.
            If None, auto-inferred using ordinal ANY dimensions.
        ishape : Shape, optional
            The input shape of the matrix.
            If None, auto-inferred using ordinal ANY dimensions.
        oshape : Shape, optional
            The output shape of the matrix.
            If None, auto-inferred using ordinal ANY dimensions.
        broadcast_dims : list, optional
            A list of the dimensions of weight that are intended to be broadcasted over the input.
            As such, they are excluded from splitting.

        Examples
        --------
        Simple usage with automatic shape inference:

        >>> W = torch.randn(5, 3)
        >>> A = Dense(W)  # Auto-infers shapes
        >>> x = torch.randn(3)
        >>> y = A(x)  # Equivalent to W @ x

        Batched usage:

        >>> W = torch.randn(2, 5, 3)  # (batch, M, N)
        >>> A = Dense(W)
        >>> x = torch.randn(2, 3)  # (batch, N)
        >>> y = A(x)  # Batched matmul

        Explicit usage (original API):

        >>> W = torch.randn(5, 3)
        >>> A = Dense(W, ("M", "N"), ("N",), ("M",))
        """
        # Auto-inference logic
        if weightshape is None or ishape is None or oshape is None:
            if not (weightshape is None and ishape is None and oshape is None):
                raise ValueError(
                    "All of weightshape, ishape, oshape must be provided or all must be None"
                )
            weightshape, ishape, oshape = self._infer_dense_shapes(weight)

        if sum(dim == ELLIPSES for dim in weightshape) > 1:
            raise ValueError(
                f"weightshape may contain at most one ellipsis, got {weightshape}"
            )

        super().__init__(NS(ishape, oshape))
        self.weight = nn.Parameter(weight, requires_grad=False)
        self._shape.weightshape = weightshape

        broadcast_dims = broadcast_dims if broadcast_dims is not None else []
        self._shape.broadcast_dims = broadcast_dims

        iscompat, _ = iscompatible(self.weightshape, self.weight.shape)
        if not iscompat:
            raise ValueError(
                f"weightshape {self.weightshape} and actual weight shape (self.weight.shape) are not compatible."
            )

    @classmethod
    def from_einstr(
        cls,
        weight: Tensor,
        einstr: str,
        broadcast_dims: list | None = None,
    ) -> "Dense":
        """Construct a Dense linop from a weight tensor and einsum-style string.

        Parameters
        ----------
        weight : Tensor
            The dense matrix used for this linop.
        einstr : str
            Einsum-style string in the format "weightshape,ishape->oshape".
            Each shape is parsed using Dim() to handle camelCase splitting.
            Example: "MN,N->M" means weightshape=("M","N"), ishape=("N",), oshape=("M",)
        broadcast_dims : list, optional
            Dimensions in weightshape that should be broadcast (not sliced during split).
            Must appear in weightshape AND (ishape or oshape).

        Returns
        -------
        Dense
            A new Dense linop.

        Examples
        --------
        >>> W = torch.randn(3, 7)
        >>> A = Dense.from_einstr(W, "MN,N->M")

        >>> W = torch.randn(2, 3, 7)  # (batch, M, N)
        >>> A = Dense.from_einstr(W, "BMN,BN->BM", broadcast_dims=["B"])
        """
        from ..nameddim import Dim

        # Parse einstr
        parts = einstr.replace(" ", "").split(",")
        if len(parts) != 2:
            raise ValueError(
                f"einstr must have format 'weightshape,ishape->oshape', got {einstr}"
            )

        weightshape_str = parts[0]
        rest = parts[1].split("->")
        if len(rest) != 2:
            raise ValueError(
                f"einstr must have format 'weightshape,ishape->oshape', got {einstr}"
            )

        ishape_str, oshape_str = rest

        # Parse using Dim()
        weightshape = Dim(weightshape_str)
        ishape = Dim(ishape_str)
        oshape = Dim(oshape_str)

        # Validate broadcast_dims if provided
        if broadcast_dims is not None:
            for dim in broadcast_dims:
                if dim not in weightshape:
                    raise ValueError(
                        f"broadcast_dim {dim} not in weightshape {weightshape}"
                    )
                if dim not in ishape and dim not in oshape:
                    raise ValueError(
                        f"broadcast_dim {dim} must appear in ishape or oshape"
                    )

        return cls(weight, weightshape, ishape, oshape, broadcast_dims=broadcast_dims)

    @property
    def weightshape(self) -> Shape:
        weightshape = self._shape.weightshape
        if not isinstance(weightshape, Sequence):
            raise TypeError(
                f"Expected weightshape to be a sequence but got {type(weightshape)}: {weightshape}"
            )
        return weightshape

    @property
    def broadcast_dims(self):
        return self._shape.broadcast_dims

    @property
    def forward_einstr(self):
        return f"{self.einstr(self.ishape)},{self.einstr(self.weightshape)}->{self.einstr(self.oshape)}"

    @property
    def adj_einstr(self):
        return f"{self.einstr(self.oshape)},{self.einstr(self.weightshape)}->{self.einstr(self.ishape)}"

    @staticmethod
    def einstr(arr):
        """Convert shape to einsum string, sanitizing ordinal ANYs for einops.

        Einops doesn't accept parentheses in dimension names, so we convert:
        - () → any0
        - (1) → any1
        - (2) → any2
        """
        parts = []
        for s in arr:
            s_str = str(s)
            if s_str == "()":
                parts.append("any0")
            elif s_str.startswith("(") and s_str.endswith(")"):
                # Ordinal ANY: (1), (2), etc.
                ordinal = s_str[1:-1]
                parts.append(f"any{ordinal}")
            else:
                parts.append(s_str)
        return " ".join(parts)

    @staticmethod
    def _infer_dense_shapes(weight: Tensor) -> tuple:
        """Infer Dense shapes from tensor using ordinal ANYs.

        Convention:
        - Last dimension = input dimension
        - Second-to-last = output dimension
        - All others = batch dimensions

        Returns
        -------
        weightshape, ishape, oshape
        """
        ndim = weight.ndim
        if ndim < 2:
            raise ValueError(f"Weight must be at least 2D, got {ndim}D")

        # Create ordinal ANYs for each dimension
        dims = [AnyDim(i) for i in range(ndim)]

        # Last dim = input, second-to-last = output
        input_dim = dims[-1]
        output_dim = dims[-2]
        batch_dims = dims[:-2]

        weightshape = tuple(dims)
        ishape = tuple(batch_dims) + (input_dim,)
        oshape = tuple(batch_dims) + (output_dim,)

        return weightshape, ishape, oshape

    @staticmethod
    def fn(dense, x, /):
        return einsum(x, dense.weight, dense.forward_einstr)

    @staticmethod
    def adj_fn(dense, x, /):
        return einsum(x, dense.weight.conj(), dense.adj_einstr)

    def adjoint(self):
        adj = copy(self)
        adj.weight = nn.Parameter(
            self.weight.conj(), requires_grad=adj.weight.requires_grad
        )
        adj._shape = adj._shape.H
        adj._update_suffix(adjoint=self._name is not None)
        return adj

    def normal(self, inner=None):
        """Compute the normal operator (adjoint times forward).

        Parameters
        ----------
        inner : NamedLinop, optional
            An optional inner operator to sandwich between the adjoint and
            forward. If None, consolidates two Dense operators into a single
            Dense.

        Returns
        -------
        NamedLinop
            The normal operator.

        Notes
        -----
        If inner is None, consolidate two Dense's into a single Dense
        ishape: [A B X Y]
        oshape: [C D X Y]
        wshape: [A B C D]

        Needs to become
        ishape: [A B X Y]
        oshape: [A1 B1 X Y]
        wshape: [A B A1 B1]

        New weight is attained as
        einsum(weight.conj(), weight, 'A1 B1 C D, A B C D -> A B A1 B1')

        -----
        ishape: [C A]
        oshape: [C1 A]
        wshape = [C C1]

        Needs to become
        ishape: [C A]
        oshape: [C2 A]
        wshape = [C C2]

        einsum(weight.conj(), weight, 'C1 C2, C C1 -> C C2')


        """
        new_oshape = []
        weight_conj_shape = list(deepcopy(self.weightshape))
        wdiag_shape = []
        wout_shape = []
        win_shape = []
        used_shapes = self.ishape + self.oshape
        # Make new oshape and weight shape
        # Rules:
        # New weightshape
        #   If dim appears in ishape and weightshape but not oshape -> increment
        #   If dim appears in ishape and weightshape AND oshape -> don't increment
        #   If dim doesn't appear in ishape or weightshape -> don't add it to new weightshape
        # Other rules:
        # new ishape is same as old ishape
        # new oshape is ishape but updated with new dimensions
        for dim in self.ishape:
            if dim in self.weightshape:
                if dim not in self.oshape:
                    win_shape.append(dim)
                    new_dim = dim.next_unused(used_shapes)
                    wout_shape.append(new_dim)
                else:
                    wdiag_shape.append(dim)
                    new_dim = dim
                i = weight_conj_shape.index(dim)
                weight_conj_shape[i] = new_dim
            else:
                new_dim = dim
            new_oshape.append(new_dim)

        if config.inner_not_relevant(inner):
            # Consolidate dense and dense adjoint into single dense
            new_weight_shape = wdiag_shape + wout_shape + win_shape
            einstr = shapes2einstr(
                self.weightshape,
                weight_conj_shape,
                new_weight_shape,
            )
            new_weight = einsum(self.weight, self.weight.conj(), einstr)
            normal = type(self)(
                new_weight,
                tuple(new_weight_shape),
                self.ishape,
                new_oshape,
            )
            normal._name = self._name
            normal._update_suffix(normal=self._name is not None)
            return normal

        assert inner is not None
        # Flexibly match shapes even through
        # ellipses and changing dimensions.
        pre = copy(self)
        pre.oshape = align(pre.oshape, inner.ishape)
        post = self.adjoint()  # Copy happens inside adjoint
        post.oshape = new_oshape
        post.ishape = align(post.ishape, inner.oshape)
        normal = post @ inner @ pre
        return normal

    @staticmethod
    def split(dense, tile):
        ibatch = tuple(tile.get(dim, slice(None)) for dim in dense.ishape)
        obatch = tuple(tile.get(dim, slice(None)) for dim in dense.oshape)
        weight = dense.split_weight(ibatch, obatch, dense.weight)
        out = copy(dense)
        out.weight = nn.Parameter(weight, requires_grad=dense.weight.requires_grad)
        return out

    def split_weight(self, ibatch, obatch, /, weight):
        weightbatch = [slice(None)] * len(self.weightshape)
        for dim, batch in zip(self.ishape, ibatch):
            if dim in self.weightshape and dim not in self.broadcast_dims:
                weightbatch[self.weightshape.index(dim)] = batch
        for dim, batch in zip(self.oshape, obatch):
            if dim in self.weightshape and dim not in self.broadcast_dims:
                weightbatch[self.weightshape.index(dim)] = batch
        return weight[tuple(weightbatch)]

    def size(self, dim: str):
        if dim in self.broadcast_dims or dim == ELLIPSES:
            return None
        # Guaranteed only one ellipsis axis
        # However it may span multiple weight axes
        extra_ellipsis_axes_count = self.weight.ndim - len(self.weightshape)
        seen_ellipsis = False
        for axis, weight_dim in enumerate(self.weightshape):
            if weight_dim == ELLIPSES:
                seen_ellipsis = True
            elif weight_dim == dim:
                if seen_ellipsis:
                    axis += extra_ellipsis_axes_count
                return self.weight.shape[axis]
        return None

__init__

__init__(
    weight: Tensor,
    weightshape: Shape | None = None,
    ishape: Shape | None = None,
    oshape: Shape | None = None,
    broadcast_dims: list | None = None,
)
PARAMETER DESCRIPTION
weight

The dense matrix used for this linop.

TYPE: Tensor

weightshape

The shape of the matrix, in symbolic form. If None, auto-inferred using ordinal ANY dimensions.

TYPE: Shape DEFAULT: None

ishape

The input shape of the matrix. If None, auto-inferred using ordinal ANY dimensions.

TYPE: Shape DEFAULT: None

oshape

The output shape of the matrix. If None, auto-inferred using ordinal ANY dimensions.

TYPE: Shape DEFAULT: None

broadcast_dims

A list of the dimensions of weight that are intended to be broadcasted over the input. As such, they are excluded from splitting.

TYPE: list DEFAULT: None

Examples:

Simple usage with automatic shape inference:

>>> W = torch.randn(5, 3)
>>> A = Dense(W)  # Auto-infers shapes
>>> x = torch.randn(3)
>>> y = A(x)  # Equivalent to W @ x

Batched usage:

>>> W = torch.randn(2, 5, 3)  # (batch, M, N)
>>> A = Dense(W)
>>> x = torch.randn(2, 3)  # (batch, N)
>>> y = A(x)  # Batched matmul

Explicit usage (original API):

>>> W = torch.randn(5, 3)
>>> A = Dense(W, ("M", "N"), ("N",), ("M",))
Source code in src/torchlinops/linops/dense.py
def __init__(
    self,
    weight: Tensor,
    weightshape: Shape | None = None,
    ishape: Shape | None = None,
    oshape: Shape | None = None,
    broadcast_dims: list | None = None,
):
    """
    Parameters
    ----------
    weight : Tensor
        The dense matrix used for this linop.
    weightshape : Shape, optional
        The shape of the matrix, in symbolic form.
        If None, auto-inferred using ordinal ANY dimensions.
    ishape : Shape, optional
        The input shape of the matrix.
        If None, auto-inferred using ordinal ANY dimensions.
    oshape : Shape, optional
        The output shape of the matrix.
        If None, auto-inferred using ordinal ANY dimensions.
    broadcast_dims : list, optional
        A list of the dimensions of weight that are intended to be broadcasted over the input.
        As such, they are excluded from splitting.

    Examples
    --------
    Simple usage with automatic shape inference:

    >>> W = torch.randn(5, 3)
    >>> A = Dense(W)  # Auto-infers shapes
    >>> x = torch.randn(3)
    >>> y = A(x)  # Equivalent to W @ x

    Batched usage:

    >>> W = torch.randn(2, 5, 3)  # (batch, M, N)
    >>> A = Dense(W)
    >>> x = torch.randn(2, 3)  # (batch, N)
    >>> y = A(x)  # Batched matmul

    Explicit usage (original API):

    >>> W = torch.randn(5, 3)
    >>> A = Dense(W, ("M", "N"), ("N",), ("M",))
    """
    # Auto-inference logic
    if weightshape is None or ishape is None or oshape is None:
        if not (weightshape is None and ishape is None and oshape is None):
            raise ValueError(
                "All of weightshape, ishape, oshape must be provided or all must be None"
            )
        weightshape, ishape, oshape = self._infer_dense_shapes(weight)

    if sum(dim == ELLIPSES for dim in weightshape) > 1:
        raise ValueError(
            f"weightshape may contain at most one ellipsis, got {weightshape}"
        )

    super().__init__(NS(ishape, oshape))
    self.weight = nn.Parameter(weight, requires_grad=False)
    self._shape.weightshape = weightshape

    broadcast_dims = broadcast_dims if broadcast_dims is not None else []
    self._shape.broadcast_dims = broadcast_dims

    iscompat, _ = iscompatible(self.weightshape, self.weight.shape)
    if not iscompat:
        raise ValueError(
            f"weightshape {self.weightshape} and actual weight shape (self.weight.shape) are not compatible."
        )

normal

normal(inner=None)

Compute the normal operator (adjoint times forward).

PARAMETER DESCRIPTION
inner

An optional inner operator to sandwich between the adjoint and forward. If None, consolidates two Dense operators into a single Dense.

TYPE: NamedLinop DEFAULT: None

RETURNS DESCRIPTION
NamedLinop

The normal operator.

Notes

If inner is None, consolidate two Dense's into a single Dense ishape: [A B X Y] oshape: [C D X Y] wshape: [A B C D]

Needs to become ishape: [A B X Y] oshape: [A1 B1 X Y] wshape: [A B A1 B1]

New weight is attained as einsum(weight.conj(), weight, 'A1 B1 C D, A B C D -> A B A1 B1')


ishape: [C A] oshape: [C1 A] wshape = [C C1]

Needs to become ishape: [C A] oshape: [C2 A] wshape = [C C2]

einsum(weight.conj(), weight, 'C1 C2, C C1 -> C C2')

Source code in src/torchlinops/linops/dense.py
def normal(self, inner=None):
    """Compute the normal operator (adjoint times forward).

    Parameters
    ----------
    inner : NamedLinop, optional
        An optional inner operator to sandwich between the adjoint and
        forward. If None, consolidates two Dense operators into a single
        Dense.

    Returns
    -------
    NamedLinop
        The normal operator.

    Notes
    -----
    If inner is None, consolidate two Dense's into a single Dense
    ishape: [A B X Y]
    oshape: [C D X Y]
    wshape: [A B C D]

    Needs to become
    ishape: [A B X Y]
    oshape: [A1 B1 X Y]
    wshape: [A B A1 B1]

    New weight is attained as
    einsum(weight.conj(), weight, 'A1 B1 C D, A B C D -> A B A1 B1')

    -----
    ishape: [C A]
    oshape: [C1 A]
    wshape = [C C1]

    Needs to become
    ishape: [C A]
    oshape: [C2 A]
    wshape = [C C2]

    einsum(weight.conj(), weight, 'C1 C2, C C1 -> C C2')


    """
    new_oshape = []
    weight_conj_shape = list(deepcopy(self.weightshape))
    wdiag_shape = []
    wout_shape = []
    win_shape = []
    used_shapes = self.ishape + self.oshape
    # Make new oshape and weight shape
    # Rules:
    # New weightshape
    #   If dim appears in ishape and weightshape but not oshape -> increment
    #   If dim appears in ishape and weightshape AND oshape -> don't increment
    #   If dim doesn't appear in ishape or weightshape -> don't add it to new weightshape
    # Other rules:
    # new ishape is same as old ishape
    # new oshape is ishape but updated with new dimensions
    for dim in self.ishape:
        if dim in self.weightshape:
            if dim not in self.oshape:
                win_shape.append(dim)
                new_dim = dim.next_unused(used_shapes)
                wout_shape.append(new_dim)
            else:
                wdiag_shape.append(dim)
                new_dim = dim
            i = weight_conj_shape.index(dim)
            weight_conj_shape[i] = new_dim
        else:
            new_dim = dim
        new_oshape.append(new_dim)

    if config.inner_not_relevant(inner):
        # Consolidate dense and dense adjoint into single dense
        new_weight_shape = wdiag_shape + wout_shape + win_shape
        einstr = shapes2einstr(
            self.weightshape,
            weight_conj_shape,
            new_weight_shape,
        )
        new_weight = einsum(self.weight, self.weight.conj(), einstr)
        normal = type(self)(
            new_weight,
            tuple(new_weight_shape),
            self.ishape,
            new_oshape,
        )
        normal._name = self._name
        normal._update_suffix(normal=self._name is not None)
        return normal

    assert inner is not None
    # Flexibly match shapes even through
    # ellipses and changing dimensions.
    pre = copy(self)
    pre.oshape = align(pre.oshape, inner.ishape)
    post = self.adjoint()  # Copy happens inside adjoint
    post.oshape = new_oshape
    post.ishape = align(post.ishape, inner.oshape)
    normal = post @ inner @ pre
    return normal