LABTITLE=: 'Files'
LABAUTHOR=: 0 : 0
Don Guinn
donguinn@gmail.com
)
LABCOMMENTS=: 0 : 0
Converted from J6. Some sections dropped from J6 as they are no longer in J7 or are quite different. Those sections are not really necessary in the understanding of files.
This is one lab which will need revision as J7 evolves. This is a first cut.
Tested in JGTK and JHS under VISTA 32 and UBUNTU 64.
AUTOLAB_jlab_ is tested in this script and left as is. It is only set to zero nullifying the code in (1) 2. It is only referenced in this lab. I do not know what it is for and leaving it alone.
)
LABERRORS=: 1
LABFOCUS=: 0
NB. =========================================================
Lab Chapter Introduction
NB. =========================================================
Lab Section Introduction
J provides a number of utility verbs (functions) to aid in the processing of files. Several file organizations are supported by J. These include text files, component files and memory mapped files. This lab is for working with text files. These text files could also be called flat files. The verbs in this lab treat a file as if it were a character string containing any of the possible 256 bit configurations found in a byte.
Also available are labs on Mapped Files and database.
In J7 the file verbs are included as part of the base. It is not necessary to "load 'files'" although it does not hurt.
)
load 'files' NB. Unnecessary in J7
NB. =========================================================
Lab Section
Most of these verbs use the foreign conjunction 1!: to access files and provide services similar to those provided by DOS. For more information on the facilities of 1!: refer to Help/Foreign Conjunction.
This lab is divided into several chapters. You may run through the entire lab or skip to a particular chapter via Studio/Chapters on the Menu bar. Go to the Summary chapter for a brief description of the files verbs.
The filename 'labfile.txt' is used for most examples and is created in the current directory. Make sure this will not destroy an important file.
)
PREPARE
3 : 0''
if. AUTOLAB_jlab_ do.
fview_z_=: fselect_z_=: ]
end.
empty''
)
PREPARE
NB. =========================================================
Lab Chapter General Rules
NB. =========================================================
Lab Section General Rules
There are a few general rules that apply to the file verbs:
* Where a file name is required as an argument, it should be a single name given either as an open or boxed character string, or as a file handle created by 1!:21 as in the following:
fread 'c:\config.sys'
fread <'c:\config.sys'
fread 270802
* Verbs that read files with a right argument of only a file name read the entire file. If a file name is linked with 1 or 2 numbers, they are the index and the size of the read. The index may be negative, and if so counts backwards from the end of the file. If size is missing, the read is to the end of the file.
For example:
fread 'labfile.txt';20 50
reads 50 bytes of the file after skipping the first 20.
* All verbs can return _1 if an error occurs.
* Verbs that write to a file return a numeric result of the number of characters written if the write is successful, or if an error, a character error message, or _1.
)
NB. =========================================================
Lab Section
* "String" verbs, like freads, fappends, and fwrites, convert text strings between Host and J formats. Text strings may be delimited by the CRLF pair, or by isolated CR or LF characters. J text strings are delimited by the LF character alone. When converting J text strings to Host format, the LF characters are replaced by CRLF, CR or LF as appropriate.
* Filenames conform to the normal rules for the base operating system.
* Extended filenames are supported. The extended name should not be enclosed in double quotes (").
)
NB. Test your system to see what your line separator is.
'abc' fwrites 'labfile.txt'
eol=:;('CR';'LF')#~(CR,LF)e.fread 'labfile.txt';_2
'Your host line separator is ',eol,'.'
NB. Some interesting names in the z locale
(<a.)i.&.>LF;CR;CRLF
NB. =========================================================
Lab Section
As you are probably aware, UNIX uses "/" for path separators where Windows uses "\". The file foreign conjunctions convert both to the one appropriate for your system; however, you can use the verb "jhostpath" to make displayed filenames correct for your system. This may be necessary for filenames passed to other applications.
You may use either "/" or "\" without regard to your base operating system. This lab will use the UNIX convention for filenames.
)
NB. See what your path separator is.
jpathsep 'dir1/dir2\dir3'
NB. =========================================================
Lab Chapter Reads
NB. =========================================================
Lab Section fread
There are 3 verbs supplied for reading files: fread, freadr and freads.
The format of fread is as follows:
[opt] fread filename[;start[,size]]
The filename, optional start and size are as described in the rules chapter and their units are characters (bytes).
If there is no left argument the file contents are returned as a character vector. If opt is supplied then fread checks the contents of the file for CR, LF or CRLF pairs and will treat them as line separators. opt must be one of the following 3 characters:
b return the result as a boxed list of lines
m return result as a matrix with each line a row
s convert CR or CRLF to LF
If the left argument is not present or null then the data is read with no conversion (byte mode).
)
NB. =========================================================
Lab Section
We will use various file verbs to look at a configuration file. The name of the file is in cfgfile.
A file name preceeded by a tilde (~) represents relative file names. The verb "jpath" translates the relative name to the full path name. "jpath" also converts both "\" and "/" to that used by the system. Some, but not all, verbs include "jpath" in their definition. If no tilde is in the name the name is returned unaltered. So it never hurts to apply "jpath" to a file name.
Let's look at the first 80 bytes of the file.
)
PREPARE
3 : 0 ''
if. 0=#fdir f=.'~config/base.cfg' do.
f=. '~system/extras\config\stdcfg.ijs'
end.
0!:101 ']cfgfile=:jpath ''',f,''''
)
PREPARE
fread (cfgfile);0 80
$fread (cfgfile);0 80
NB. =========================================================
Lab Section
The file contains line separators. The display of the results is in multiple lines; however, the returned result is a vector.
The line separator characters are retained in the data.
Search for line separators in the string:
)
PREPARE Do my own write to session so appropriate eol is used.
eol=:;('CR';'LF')#~(CR,LF)e.toHOST LF
0!:101]eol,' ss fread (cfgfile);0 80'
PREPARE _________________________
NB. =========================================================
Lab Section
The 'b' option boxes each line of the file.
)
'b' fread (cfgfile);0 80
NB. =========================================================
Lab Section
The 'm' option creates a matrix with each line in the file on a separate row. The line separators are removed.
Notice that lines may be padded with blanks.
)
'm' fread (cfgfile);0 80
$'m' fread (cfgfile);0 80
NB. =========================================================
Lab Section freadr
freadr reads records from flat file consisting of fixed length records. The records are delimited by one (only) of CR, LF, or CRLF.
The result is a matrix of records excluding the separator.
For freadr start and count refer to records, not characters.
For example:
freadr 'xxx';4 2
reads 2 records after skipping 4 records.
A temporary file called "labfile.txt" is generated to illustrate.
)
(":i.5 4) fwrites 'labfile.txt'
freadr 'labfile.txt'
$freadr 'labfile.txt'
NB. =========================================================
Lab Section
The optional index and length specify the line (item) index and number of lines to read.
)
freadr 'labfile.txt';1 3
NB. =========================================================
Lab Section
Be careful using freadr. If the file does not contain fixed length records unpredictable results may occur which may result in an error or garbage data.
Deleting "labfile.txt" to clean up.
)
ferase 'labfile.txt'
NB. =========================================================
Lab Section freads
freads reads characters from a file, converting any CRLF or isolated CR characters into LF characters as J keeps script file internally. It is equivalent to 's'&fread.
)
freads (cfgfile);0 80
]l=.$fread cfgfile
]ls=.$'s' fread cfgfile
l-ls NB. The result will be zero for UNIX
NB. =========================================================
Lab Chapter Writes
NB. =========================================================
Lab Section Writes
There are 5 verbs that write to files: fappend, fappends, freplace, fwrite and fwrites. They are all of the form:
data verb filename[;number]
The data for all writes must be text (bytes). Text arrays of higher rank will be raveled before writing.
As with reads, the file can be referred to by name or by a file handle created by 1!:21.
)
NB. =========================================================
Lab Section fwrite
fwrite writes character text to a file. The text is first raveled. Except for raveling, the text is unaltered and can contain any bit configuration in a character (byte). If the file already exists, it is overwritten. If the file does not exist it is created.
If the write is successful, it returns the number of characters written. Errors can be indicated by returning a character error message or _1.
)
(2 3$'abcdef') fwrite 'labfile.txt'
fread 'labfile.txt'
NB. =========================================================
Lab Section fwrites
fwrites is like fwrite except the host line separator is appended to the end of each each line. Any CR or LF characters in the data are converted to the host line separator.
)
(2 3$'abcdef') fwrites 'labfile.txt'
fread 'labfile.txt'
a.i. fread 'labfile.txt'
NB. =========================================================
Lab Section fappend
fappend appends the text to the end of the file. If the file does not exist or is empty then fappend is like fwrite.
If the file does not exist the file is created.
)
'abc' fwrite 'labfile.txt'
'xy' fappend 'labfile.txt'
fread 'labfile.txt'
NB. =========================================================
Lab Section fappends
fappends is like fappend except it handles the line separators CRLF, CR and LF as in fwrites. Notice that fappends does not start a new line.
If the file does not exist the file is created.
)
'abc' fwrite 'labfile.txt'
'xy' fappends 'labfile.txt'
'uvw' fappends 'labfile.txt'
fread 'labfile.txt'
a.i. fread 'labfile.txt'
NB. =========================================================
Lab Section freplace
freplace replaces text in a file. The filename is boxed and followed by the position in the file to start replacing. There is no checking for line separators in the text or in the characters in the file replaced.
If the starting index is negative then the replacement start is from the end of the file.
If the number of characters is such that some characters would extend beyond the end of the file, it is extended.
)
(10$'a') fwrite 'labfile.txt'
'XYZ' freplace 'labfile.txt';4
fread 'labfile.txt'
NB. =========================================================
Lab Section
If the starting index is beyond the end of the file the file is extended with nulls before replacement.
)
(10$'a') fwrite 'labfile.txt'
'XYZ' freplace 'labfile.txt';15
a.i. fread 'labfile.txt'
NB. =========================================================
Lab Chapter Directory and File Management
NB. =========================================================
Lab Section Directory and File Management
This chapter covers some directory and file management verbs. These verbs are very similar to those found in DOS. The list of verbs described below is far from complete. J7 has a number of new verbs for managing files. Following are a few of interest.
Explore '~system/main/stdlib.ijs' to find others. If you feel it is of general use, do not hesitate to add them to this tutorial.
)
NB. =========================================================
Lab Section fdir
fdir lists the directory information for files and directories matching the filename. The filename can contain wildcard characters.
The returned value is a boxed matrix, a row for each file found and 5 columns as follows: filename, date and time of file creation in J time stamp format, file size in bytes, access privileges, flags.
)
fdir jpath '~system\main\s*.ijs'
NB. =========================================================
Lab Section
dir is similar to fdir except it takes an optional left argument and makes a prettier return as follows:
- if not given, defaults to 'n'
- if character, returns a formatted directory,
where x is the sort option:
d=by date
n=by name
s=by size
- if numeric, there are 1 or 2 elements:
0{ 0= list short names
1= boxed list of full pathnames
2= full directory list
1{ 0= filenames only (default)
1= include subdirectories
)
's' dir jpath '~system\main\s*.ijs'
NB. =========================================================
Lab Section ferase
ferase erases a file or all matching files if wildcards are included in the filename. If the erase is successful a 1 is returned for each file erased. _1 is returned for each file that could not be erased.
)
ferase 'labfile.txt'
ferase 'labfile.txt'
NB. =========================================================
Lab Section fexist
fexist returns 1 if the file exists, 0 if not.
)
'abc' fwrite'labfile.txt'
fexist 'labfile.txt'
ferase 'labfile.txt'
fexist 'labfile.txt'
NB. =========================================================
Lab Section fsize
fsize returns the size of a file in bytes. By the way, this is the script file that defines the file verbs.
)
PREPARE
0!:101 'fsize ''',(;(4!:4<'fread'){4!:3''),''''
PREPARE
NB. =========================================================
Lab Section fstamp
fstamp returns the date and time a file is created in J timestamp format.
)
PREPARE
0!:101 'fstamp ''',(;(4!:4<'fread'){4!:3''),''''
PREPARE
NB. =========================================================
Lab Section fss
fss performs a string search on a file returning the indices where the string is found.
)
]t=:'tab' fss jpath '~system\main\stdlib.ijs'
fread (jpath '~system\main\stdlib.ijs');(_29+{.t),40
NB. =========================================================
Lab Chapter Session Input and Output
NB. =========================================================
Lab Section Session Input and Output
The J session is displayed in an execution window. As you well know by now, you can type sentences into an execution window and the results of that sentence will be displayed in it. Like on a typewriter.
This typewriter-type interface was the only way to interactively communicate with computers before video screens were available. It is an easy interface for simple applications.
There are five pre-defined file handles. They may be used with fread and fwrite.
1 read from the keyboard
3 read from standard input (stdin)
2 write to screen
4 write to standard output (stdout)
5 write to standard error (stderr)
smoutput writes to the screen as well.
For serious manipulation of the session window, study the jfe interface.
)
smoutput 'This is some text'
NB. =========================================================
Lab Chapter Miscellaneous
NB. =========================================================
Lab Section fview
fview displays the file using a Windows edit box. The right argument is defined as for fread. The display is asynchronous, that is, the J window is unlocked and J continues to execute. The following is a view of the files script that defines the verbs just covered. Why not just browse it for a while?
At this time fview is only available in JGTK.
)
fview jpath '~system\main\stdlib.ijs'
NB. =========================================================
Lab Section fcopynew
fcopynew is a copy/merge utility.
Form: tofile fcopynew fromfiles
fromfiles must be a single file name or a boxed list of files. If tofile does not exist it will be created.
The files specified in fromfiles are read and raveled together and compared to the contents of tofile. If they differ or newfile does not exist then the raveled contents of the files are written to newfile.
Returns:
No change. newfile matches: 0, size of newfile
Contents changed. Copied: 1, size of newfile
Failure. No copy done: _1
)
NB. =========================================================
Lab Section fssrplc
fstringrpelace (fssrplc for a shorter name) search for occurrences of a specified string in the file and replaces those occurrences with another string. The format of fssrplc is:
(oldstring;newstring) fssrplc (filename|file handle)
The old string and new string may be of different lengths.
)
'abcdefcdgh' fwrite 'labfile.txt'
('cd';'XYZ') fssrplc 'labfile.txt'
fread 'labfile.txt'
NB. =========================================================
Lab Section open and close
The Introduction mentioned that file handles could be used in place of file names for some file verbs. Files can be opened once, manipulated using the file handle for many file accesses, then closed when finished. An advantage of this approach is performance. Opening and closing files over and over does entail some overhead.
Another advantage is files, or parts of files, can be locked to prevent other applications updating a file while your application is updating it. See 1!:30 and 1!:31 for using locks.
Be careful about leaving files open for extended periods. You cannot be guaranteed that all data has been written to disk until the file is closed. Other applications making legitimate accesses to the file may be blocked.
If you wish, you can create your own cover verbs, fopen and fclose, as follows:
)
fopen_z_=:1!:21@boxopen
fclose_z_=:1!:22@boxopen
NB. Placed in z locale to go with other file verbs.
NB. =========================================================
Lab Section fopen and fclose
fopen will return a file handle if successful or raise an error condition if not.
fclose returns 1 if successful or raises the error condition.
)
]h=:fopen 'labfile.txt'
'abc' fwrite h
'defg' fappend h
fread h
fclose h
NB. =========================================================
Lab Section
Maybe you noticed something strange? The result does not look right?
For some reason the file size is not reset if a fwrite to an existing file is shorter than the file size.
If it does look right then the bug has been fixed.
To be on the safe side it would be advisable to erase a file instead of just replacing it.
)
NB. =========================================================
Lab Section Other file verbs
Another interesting verb available in 1!: is:
1!:20 File numbers and names of open files
)
]h=:fopen 'labfile.txt';jpath '~system\main\files.ijs'
1!:20 ''
fclose"0 h
1!:20 ''
NB. =========================================================
Lab Chapter Relative File Names
NB. =========================================================
Lab Section Overview
File names can be specified in several forms. You can give the full path name or a relative name. Redirection and wildcards available in your operating system may be used as well.
The verb jpath allows one to find the various J libraries with relative reference. Using jpath an application need not specify the full path to a library. System libraries are defined in the profile, and user libraries in the J Configuration under the Folders category. You may modify or add user libraries to the J system there.
jpath with an argument with a leading ~ looks up the name in the folder tables. The name is case sensitive.
Here is the full filename to the profile script which starts J.
)
jpath '~system/profile.ijs' NB. The default profile.
NB. =========================================================
Lab Section Nouns and Verbs to find the Libraries
The standard J profile creates two nouns to define the the J libraries, SystemFolders_j_ and UserFolders_j_. You can edit UserFolders_j_ with Edit/Configure/Folders.
Changes take effect in the next restart. See the remarks in the ~config/folders.cfg .
)
NB. The relative names for various J system libraries are:
>{."1 SystemFolders_j_
NB. What`s in the user library?
fdir jpath '~user/*.*'
NB. =========================================================
Lab Section The J Profile
A couple more interesting names in the z locale:
)
ARGV NB. The command line starting J.
BINPATH NB. The path to the J binaries.
NB. =========================================================
Lab Section
There is one more directory of interest - the current working directory. Initially it is the same as the J ~bin; however, it can be specified to be somewhere else by the script or icon starting J and can be changed by a J application.
It is managed by the foreign conjunctions 1!:43 to query the current working directory and 1!:44 to set it.
)
1!:43''
NB. =========================================================
Lab Chapter Summary
NB. =========================================================
Lab Section Summary
A summary of the primary file verbs:
dat fappend fl append
dat fappends fl append string
to fcopynew fls copy files (if changed)
fdir dirpath file directory
ferase fl erase file
fexist fl return 1 if file exists
opt fread fl read file
freadr fl read records (flat file)
freads fl read string
dat freplace fl replace in file
fsize fl size of file
str fss fl string search file
oldnew fssrplc fl search and replace in file
fstamp fl file timestamp
fview fl view file
dat fwrite fl write file
dat fwrites fl write string
fl - a single filename or handle
fls - one or more file names, if more than 1 then boxed
dirpath - a directory path, may include wildcards
dat - data to write to a file
to - destination for copy files
opt - options for how a verb works
oldnew - old and new for string replacements in files
)
PREPARE Clean up.
ferase 'labfile.txt'
PREPARE ___________ 