File I/O Enhancements for J Source

These patches add new file I/O capabilities to J through three feature groups: readline functions, virtual (RAM-based) files, and character-level I/O control.

Note: Most patches are standalone. Only virtual-files requires readline-functions. See Applying the Patches for details.

Readline Functions (1!:13, 1!:14, 1!:15)

Patch: readline-functions.patch

Overview

These functions provide line-by-line file reading, complementing J’s existing block-based file operations:

Function Description Newline Handling
1!:13 Read one line Includes newline
1!:14 Read one line (trimmed) Strips CR/LF
1!:15 Peek one line Strips CR/LF, doesn’t advance position

Usage Examples

NB. Open a file for reading
f =. 1!:21 <'data.txt'

NB. Read line by line (includes newline)
line1 =. 1!:13 f
line2 =. 1!:13 f

NB. Read line without trailing newline
clean_line =. 1!:14 f

NB. Peek at next line without consuming it
next =. 1!:15 f
same =. 1!:15 f   NB. Returns same content

NB. Close file
1!:22 f

Processing a File Line by Line

process_file =: 3 : 0
  f =. 1!:21 <y
  result =. ''
  while. 0 < # line =. 1!:14 f do.
    result =. result , process_line line
  end.
  1!:22 f
  result
)

Technical Details

The implementation uses C’s fgets() to read lines up to 1000 characters. For regular files:

Special file handles are supported:

Virtual Files (1!:60)

Patch: virtual-files.patch

Overview

Virtual files exist entirely in RAM and behave like regular files. They’re useful for: - Temporary data without filesystem overhead - Testing file-handling code - In-memory buffers with file semantics - Inter-component communication within J

Creating Virtual Files

NB. Create with automatic name
vf =. 1!:60 ''

NB. Create with custom name (for debugging)
vf =. 1!:60 'mybuffer'

NB. The returned handle (>= 1000000) works with all file operations

Usage Examples

NB. Create a virtual file
vf =. 1!:60 'test'

NB. Write data
'Hello, World!' 1!:2 vf

NB. Read it back
data =. 1!:1 vf    NB. Returns: Hello, World!

NB. Append more data
' More text.' 1!:3 vf

NB. Check size
size =. 1!:4 vf    NB. Returns: 24

NB. Read line by line
1!:14 vf           NB. Works with readline functions too!

Technical Implementation

Virtual files use a custom VirtFile structure:

typedef struct {
    C *data;              // Content buffer (auto-grows)
    I size;               // Current content size
    I capacity;           // Allocated buffer size
    I position;           // Read/write position
    I magic;              // 0x56465249 ('VFIR') identifies virtual files
    I virtid;             // Unique ID >= 1000000
    pthread_rwlock_t lock; // Thread safety
} VirtFile;

Key features:

Limitations

Character I/O Control (1!:111)

Patch: char-io-control.patch

Overview

Provides character-level terminal control for building interactive applications:

Mode Description
(1!:111) 1 Read character with echo
(1!:111) 0 Enter silent mode / read without echo
(1!:111) _1 Restore normal terminal
(1!:111) 2 Flush stdout immediately

Note: Parentheses are required due to J’s parsing of 3-digit numbers.

Usage Examples

NB. Read single character with echo (like normal typing)
char =. (1!:111) 1

NB. Enter silent mode (for password input, etc.)
(1!:111) 0     NB. First call enters silent mode
c1 =. (1!:111) 0     NB. Subsequent calls read silently
c2 =. (1!:111) 0
(1!:111) _1    NB. Restore normal echo

NB. Immediate output (essential for character-by-character display)
'Press any key: ' 1!:2 (4)   NB. Write to stdout
(1!:111) 2                    NB. Flush - text appears immediately
key =. (1!:111) 1             NB. Wait for keypress

Building a Simple Menu

menu =: 3 : 0
  echo 'Choose an option:'
  echo '  [a] Add item'
  echo '  [d] Delete item'
  echo '  [q] Quit'
  'Choice: ' 1!:2 (4)
  (1!:111) 2                    NB. Flush prompt
  choice =. (1!:111) 1          NB. Get single keypress with echo
  echo ''                       NB. Newline after choice
  choice
)

Technical Implementation

The function uses platform-specific APIs:

Unix/Linux (termios):

// Disable canonical mode and echo
new_tio.c_lflag &= ~ICANON;  // Character-at-a-time input
new_tio.c_lflag &= ~ECHO;    // Silent mode
tcsetattr(STDIN_FILENO, TCSANOW, &new_tio);

// Read single character
read(STDIN_FILENO, &c, 1);

Windows (conio.h):

c = _getch();   // Silent read
c = _getche();  // Read with echo

Global state management ensures: - Terminal settings are saved before modification - Silent mode persists across multiple reads - Original settings are restored on (1!:111) _1

Applying the Patches

Patch Dependencies

readline-functions.patch     (standalone)
virtual-files.patch          (requires readline-functions)
char-io-control.patch        (standalone)

Note: Only virtual-files has a functional dependency on readline (it extends readline to work with RAM-based files). The other patches are independent.

Readline Functions

cd /path/to/jsource
git apply readline-functions.patch

Standalone patch - can be applied independently.

Virtual Files (requires readline)

cd /path/to/jsource
git apply readline-functions.patch
git apply virtual-files.patch

Virtual files extends the readline functions to work with RAM-based files.

Character I/O

cd /path/to/jsource
git apply char-io-control.patch

Standalone patch - can be applied independently.

All File I/O Enhancements

When applying all patches together, apply in this order:

cd /path/to/jsource
git apply downloads/file-io-enhancements/readline/readline-functions.patch
git apply downloads/file-io-enhancements/virtual-files/virtual-files.patch
git apply downloads/file-io-enhancements/char-io/char-io-control.patch

Or apply just the ones you need - readline and char-io work independently

Summary Table

Function Purpose Patch
1!:13 Read line (with newline) readline-functions
1!:14 Read line (trimmed) readline-functions
1!:15 Peek line readline-functions
1!:60 Create virtual file virtual-files
(1!:111) 0/1/_1/2 Character I/O control char-io-control