Requests for J system extensions and enhancements in 2007 (2005, 2006, current).

Contents

  1. i. with ordered x
  2. optimize f~/~
  3. make f^:_1: invertible
  4. make x&(]\) invertible with ,/
  5. optimization of dyad {.@:/:
  6. custom security levels

i. with ordered x

I often use dyadic i. when the left argument x is sorted, or might as well be sorted. It may then be much more efficient to use the following function iord, or one like it: 

iord=: 4 : 0
k=. x I. y
(#x) (I. -. y -:"_1 k{x)} k
)

NB. example of use
   ts=: 6!:2 , 7!:2@]
   XV=: /:~ ~. ?. 1e6 20$10
   YV=: (, |."1) (2000 ?. 1e6) { XV   NB. iord must work when some y-items aren't in x
   XV (i. -: iord) YV                 NB. verify iord works ok
1
   ts 'XV i. YV'
0.68132 8.40595e6
   ts 'XV iord YV'
0.0103253 600320

Is it feasible to define i.!._1 (say) to mean 'i. with the assumption that x is ordered (up or down, cf I.)'? If not, I hope others find the above idea useful anyway! -- EwartShaw 2007-01-22 16:18:01


optimize f~/~

The phrase x f/ y makes a x ,&# y table; but often what one desires is a y ,&# x table (particularly to make the items of the resulting table processable natively by rank-1 primitives like #.). The idioms |:@:(f/) [: |: f/ f~/~ are often used to generate such tables; it would be nice if one or more of these idioms were optimized.

-- DanBron 2007-03-08 15:49:00 

   ts=: 6!:2 , 7!:2@]

   x=: 1000 ?@$ 0
   y=: 10000 ?@$ 0

   ts 'x -/y'
0.227732 1.34219e8
   ts 'x -~/y'
0.230783 1.34219e8

   ts 'x -~/~ y'
0.209292 1.34219e8


make f^:_1: invertible

The phrase f^:_1 is invertible for invertible f. But replacing the constant _1 with the verb _1: renders the phrase uninvertible: 

   ]^:_1  b. _1
]

   ]^:_1: b. _1
|domain error
|       ]^:_1:b._1

Please make ^:_1: invertible. This will allow terser and more elegant invertible trains of the form f g^:_1: h where the leftmost token in the verb (train) h is a numeric constant, as in f g^:_1: 256 #. ] or f g^:_1: 256&#.. Note that replacing _1 with _1: permitted the elision of otherwise-mandatory parens around g^:_1.

-- DanBron 2007-03-12 17:05:34


make x&(]\) invertible with ,/

Currently, the inverse of x&(]\) for non-positive scalar integer x is undefined. I believe a reasonable inverse is ,/, as in _4&(]\) :. (,/).

As in a prior request, x ]\ ] and other likely forms should also be invertible (e.g. x&([\), x [\ ], x ]\ [, x [\ [, etc.)

I don't know what to do if x is non-scalar, as it might introduce fills. Perhaps leave it as a domain error.

For context, the motivation behind these last 3 "inversion" requests is to be able to write
   ascii85 =: (33 + [: 85&#.^:_1 (256) #.  _4&(]\) :. (,/) )&.(a.&i.)
instead as
   ascii85 =: (33 + 85 #.^:_1: 256 #. _4 ]\ ])&.(a.&i.)
and still get the identity -: ]&.ascii85 as in: 

   ascii85^:_1 ascii85 'hello world'
hello world

I've encountered these roadblocks before, in similar situations.

-- DanBron 2007-03-12 17:32:19


optimization of dyad {.@:/:

The dyad {.@:/: is an brief, obvious (to write), clean (to read) generalized equivalent to {~ (i. >./). The former has all the hallmarks of an idiom, but the latter is significantly faster.

Definitions: 

   x      =:  ?.~4e6
   y      =:  a.$~#x

   max0   =:  ({~ (i. >./))
   max1   =:  {.@:/:

Compared: 

   alg   time   space  
  ---- ------ ------- 
  max0   1.00     1.0 
  max1 544.27 43691.5

It would be nice if the idiom were optimized. At least for y amenable to >./; i.e. numeric RHAs. But it would be nicer to evade that constraint: while ;y might be numeric, y might be boxed, because it's associated 1-to-1 with non-numeric data. For example: 

   NB.  Data would normally come in to J this way, from a CSV file, database...
   XY    =:   x ,.&:(<">) y  

   max0m  =:  ({~ (i. >./)@:;)/ @:|:
   max1m  =:  {.@:/:/                  NB.  Idiom still prettier.

   (max0m -: max1m) XY
1

Other cases to consider:

  1. The special case monad {.@:(/:~) which would be equivalent to {.@:/:~ (i.e. the reflexive of the optimized {.@:/:) which is a generalization of >./ (which itself is restricted to numbers). This optimization would give J "the maximum TAO".

  2. Substitutions of \: for /: and {: for {..

  3. Possibly dyads n { /: and n {. /: for n a valid index of y.

-- DanBron 2007-04-13 19:16:22


custom security levels

It might be nice if there were a new foreign (say 9!:255) which let us customize our security level.

Examples of limitations with the current system (9!:25[1):

The request is for a foreign with ranks 0 0 0. Each scalar argument is a complex number, or, in the case or the LHA to the dyad, possibly a box.

The monadic definition is 9!:255 y =. N j. B with N a positive integer representing a foreign family, and B a boolean. The result shall be that the foreign family N!: will be prohibited or admitted as B is 0 or 1.

The dyad is defined similarly, but gives finer grained control over the specific foreigns. The definition is (x =. M j. B1) 9!:255 y =. N j. B0. Then the foreign family N!: will be prohibited or admitted as B0 is 0 or 1, with the exception that the specific foreign N!:M will be prohibited or admitted as B1 is 0 or 1.

Finally, x may be boxed, the contents of the box being an array of complex numbers with 0 < */@:$. In this case, 9!:255 be defined as (>@:[ 9!:255 ]). The restriction 0 < */@:$ is not required if the implementation is careful not to call 9!:255 on a cell of fills (in that case a: 9!:255 y would be no different from 9!:255 y. Though this may not conform to the expectation that if a frame contains 0, the verb should "not do anything").

These definitions make it possible, for example, to prohibit all of 2!: except 2!:55, with 55j1 (9!:255) 2. Furthermore, once the user customizes the security level to his taste, he could prohibit reversions with 9 (9!:255)~ 255. There is no need for 9!:255 to lock itself in, as 9!:25 must.

The security must be enforced at level of the individual foreigns (as with the current 9!:25), not at the level of the lexer/parser, to obviate the possibility of evading the prohibitions through tricks like ". 'm!:n y', ]&.": 'm!:n y', 'm!:n' 128!:2 y, (9!:29[1), 9!:27 'm!:n y' and the like.

Notes:

  1. Complex numbers were chosen over signed numbers because one wants to be able to admit m!:0. and 0!:n.

  2. The dyad's left rank of 0 was chosen because if it were 1, then one might be tempted to use (4 5, 6 7 8 9,:255) (9!:255) 1 2 9 which would result in the undesired restrictions of 1!:0, 2!:0 and 3!:0. But, since the left rank of the dyad is 0, and its argument is optionally boxed, one may instead say: (4 5 ; 6 7 8 9 ; 255) (9!:255) 1 2 9 which does what one expects.

  3. Notationally, one might prefer the argument order foriegn_family 9!:255 specific_foriegn for the dyad, because it results in sentences like 2 (9!:255) 55j1 (as opposed to 55j1 (9!:255) 2 which is reversed wrt !:). However, having the foreign family on the right is consistent with the definition of the monad, permits the tautology 9!:255 y ↔ a: 9!:255 y, and conforms to the pattern exemplified by, e.g., 1!:1 vs 1!:2.

-- DanBron 2007-04-18 00:42:32

System/Interpreter/Requests07 (last edited 2008-12-08 10:45:30 by )