FIXES revision 1.3 1 /****************************************************************
2 Copyright (C) Lucent Technologies 1997
3 All Rights Reserved
4
5 Permission to use, copy, modify, and distribute this software and
6 its documentation for any purpose and without fee is hereby
7 granted, provided that the above copyright notice appear in all
8 copies and that both that the copyright notice and this
9 permission notice and warranty disclaimer appear in supporting
10 documentation, and that the name Lucent Technologies or any of
11 its entities not be used in advertising or publicity pertaining
12 to distribution of the software without specific, written prior
13 permission.
14
15 LUCENT DISCLAIMS ALL WARRANTIES WITH REGARD TO THIS SOFTWARE,
16 INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS.
17 IN NO EVENT SHALL LUCENT OR ANY OF ITS ENTITIES BE LIABLE FOR ANY
18 SPECIAL, INDIRECT OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
19 WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER
20 IN AN ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION,
21 ARISING OUT OF OR IN CONNECTION WITH THE USE OR PERFORMANCE OF
22 THIS SOFTWARE.
23 ****************************************************************/
24
25 This file lists all bug fixes, changes, etc., made since the AWK book
26 was sent to the printers in August, 1987.
27
28 February 18, 2020:
29 Additional cleanups from Christos Zoulas. It's no longer necessary
30 to use the -y flag to bison.
31
32 February 6, 2020:
33 Additional small cleanups from Christos Zoulas. awk is now
34 a little more robust about reporting I/O errors upon exit.
35
36 January 31, 2020:
37 Merge PR #70, which avoids use of variable length arrays. Thanks
38 to GitHub user michaelforney. Fix issue #60 ({0} in interval
39 expressions doesn't work). Also get all tests working again.
40 Thanks to Arnold Robbins.
41
42 January 24, 2020:
43 A number of small cleanups from Christos Zoulas. Add the close
44 on exec flag to files/pipes opened for redirection; courtesy of
45 Arnold Robbins.
46
47 January 19, 2020:
48 If POSIXLY_CORRECT is set in the environment, then sub and gsub
49 use POSIX rules for multiple backslashes. This fixes Issue #66,
50 while maintaining backwards compatibility.
51
52 January 9, 2020:
53 Input/output errors on closing files are now fatal instead of
54 mere warnings. Thanks to Martijn Dekker <martijn@inlv.org>.
55
56 January 5, 2020:
57 Fix a bug in the concatentation of two string constants into
58 one done in the grammar. Fixes GitHub issue #61. Thanks
59 to GitHub user awkfan77 for pointing out the direction for
60 the fix. New test T.concat added to the test suite.
61 Fix a few memory leaks reported by valgrind, as well.
62
63 December 27, 2019:
64 Fix a bug whereby a{0,3} could match four a's. Thanks to
65 "Anonymous AWK fan" for the report.
66
67 December 11, 2019:
68 Further printf-related fixes for 32 bit systems.
69 Thanks again to Christos Zoulas.
70
71 December 8, 2019:
72 Fix the return value of sprintf("%d") on 32 bit systems.
73 Thanks to Jim Lowe for the report and to Christos Zoulas
74 for the fix.
75
76 November 10, 2019:
77 Convert a number of Boolean integer variables into
78 actual bools. Convert compile_time variable into an
79 enum and simplify some of the related code. Thanks
80 to Arnold Robbins.
81
82 November 8, 2019:
83 Fix from Ori Bernstein to get UTF-8 characters instead of
84 bytes when FS = "". This is currently the only bit of
85 the One True Awk that understands multibyte characters.
86 From Arnold Robbins, apply some cleanups in the test suite.
87
88 October 25, 2019:
89 More fixes and cleanups from NetBSD, courtesy of Christos
90 Zoulas. Merges PRs 54 and 55.
91
92 October 24, 2019:
93 Import second round of code cleanups from NetBSD. Much thanks
94 to Christos Zoulas (GitHub user zoulasc). Merges PR 53.
95 Add an optimization for string concatenation, also from
96 Christos.
97
98 October 17, 2019:
99 Import code cleanups from NetBSD. Much thanks to Christos
100 Zoulas (GitHub user zoulasc). Merges PR 51.
101
102 October 6, 2019:
103 Import code from NetBSD awk that implements RS as a regular
104 expression.
105
106 September 10, 2019:
107 Fixes for various array / memory overruns found via gcc's
108 -fsanitize=unknown. Thanks to Alexander Richardson (GitHub
109 user arichardson). Merges PRs 47 and 48.
110
111 July 28, 2019:
112 Import grammar optimization from NetBSD: Two string constants
113 concatenated together get turned into a single string.
114
115 July 26, 2019:
116 Support POSIX-specified C-style escape sequences "\a" (alarm)
117 and "\v" (vertical tab) in command line arguments and regular
118 expressions, further to the support for them in strings added on
119 Apr 9, 1989. These now no longer match as literal "a" and "v"
120 characters (as they don't on other awk implementations).
121 Thanks to Martijn Dekker.
122
123 July 17, 2019:
124 Pull in a number of code cleanups and minor fixes from
125 Warner Losh's bsd-ota branch. The only user visible change
126 is the use of random(3) as the random number generator.
127 Thanks to Warner Losh for collecting all these fixes in
128 one easy place to get them from.
129
130 July 16, 2019:
131 Fix field splitting to use FS value as of the time a record
132 was read or assigned to. Thanks to GitHub user Cody Mello (melloc)
133 for the fix. (Merged from his branch, via PR #42.) Updated
134 testdir/T.split per said PR as well.
135
136 June 24, 2019:
137 Extract awktest.tar into testdir directory. Add some very
138 simple mechanics to the makefile for running the tests and
139 for cleaning up. No changes to awk itself.
140
141 June 17, 2019:
142 Disallow deleting SYMTAB and its elements, which creates
143 use-after-free bugs. Thanks to GitHub user Cody Mello (melloc)
144 for the fix. (Merged from PR #43.)
145
146 June 5, 2019:
147 Allow unmatched right parenthesis in a regular expression to
148 be treated literally. Fixes Issue #40. Thanks to GitHub user
149 Warner Losh (bsdimp) for the report. Thanks to Arnold Robbins
150 for the fix.
151
152 May 29,2019:
153 Fix check for command line arguments to no longer require that
154 first character after '=' not be another '='. Reverts change of
155 August 11, 1989. Thanks to GitHub user Jamie Landeg Jones for
156 pointing out the issue; from Issue #38.
157
158 Apr 7, 2019:
159 Update awktest.tar(p.50) to use modern options to sort. Needed
160 for Android development. Thanks to GitHub user mohd-akram (Mohamed
161 Akram). From Issue #33.
162
163 Mar 12, 2019:
164 Added very simplistic support for cross-compiling in the
165 makefile. We are NOT going to go in the direction of the
166 autotools, though. Thanks to GitHub user nee-san for
167 the basic change. (Merged from PR #34.)
168
169 Mar 5, 2019:
170 Added support for POSIX-standard interval expressions (a.k.a.
171 bounds, a.k.a. repetition expressions) in regular expressions,
172 backported (via NetBSD) from Apple awk-24 (20070501).
173 Thanks to Martijn Dekker <martijn@inlv.org> for the port.
174 (Merged from PR #30.)
175
176 Mar 3, 2019:
177 Merge PRs as follows:
178 #12: Avoid undefined behaviour when using ctype(3) functions in
179 relex(). Thanks to GitHub user iamleot.
180 #31: Make getline handle numeric strings, and update FIXES. Thanks
181 to GitHub user arnoldrobbins.
182 #32: maketab: support build systems with read-only source. Thanks
183 to GitHub user enh.
184
185 Jan 25, 2019:
186 Make getline handle numeric strings properly in all cases.
187 (Thanks, Arnold.)
188
189 Jan 21, 2019:
190 Merged a number of small fixes from GitHub pull requests.
191 Thanks to GitHub users Arnold Robbins (arnoldrobbins),
192 Cody Mello (melloc) and Christoph Junghans (junghans).
193 PR numbers: 13-21, 23, 24, 27.
194
195 Oct 25, 2018:
196 Added test in maketab.c to prevent generating a proctab entry
197 for YYSTYPE_IS_DEFINED. It was harmless but some gcc settings
198 generated a warning message. Thanks to Nan Xiao for report.
199
200 Aug 27, 2018:
201 Disallow '$' in printf formats; arguments evaluated in order
202 and printed in order.
203
204 Added some casts to silence warnings on debugging printfs.
205 (Thanks, Arnold.)
206
207 Aug 23, 2018:
208 A long list of fixes courtesy of Arnold Robbins,
209 to whom profound thanks.
210
211 1. ofs-rebuild: OFS value used to rebuild the record was incorrect.
212 Fixed August 19, 2014. Revised fix August 2018.
213
214 2. system-status: Instead of a floating-point division by 256, use
215 the wait(2) macros to create a reasonable exit status.
216 Fixed March 12, 2016.
217
218 3. space: Use provided xisblank() function instead of ispace() for
219 matching [[:blank:]].
220
221 4. a-format: Add POSIX standard %a and %A to supported formats. Check
222 at runtime that this format is available.
223
224 5. decr-NF: Decrementing NF did not change $0. This is a decades-old
225 bug. There are interactions with the old and new value of OFS as well.
226 Most of the fix came from the NetBSD awk.
227
228 6. string-conv: String conversions of scalars were sticky. Once a
229 conversion to string happened, even with OFMT, that value was used until
230 a new numeric value was assigned, even if OFMT differed from CONVFMT,
231 and also if CONVFMT changed.
232
233 7. unary-plus: Unary plus on a string constant returned the string.
234 Instead, it should convert the value to numeric and give that value.
235
236 Also added Arnold's tests for these to awktest.tar as T.arnold.
237
238 Aug 15, 2018:
239 fixed mangled awktest.tar (thanks, Arnold), posted all
240 current (very minor) fixes to github / onetrueawk
241
242 Jun 7, 2018:
243 (yes, a long layoff)
244 Updated some broken tests (beebe.tar, T.lilly)
245 [thanks to Arnold Robbins]
246
247 Mar 26, 2015:
248 buffer overflow in error reporting; thanks to tobias ulmer
249 and john-mark gurney for spotting it and the fix.
250
251 Feb 4, 2013:
252 cleaned up a handful of tests that didn't seem to actually
253 test for correct behavior: T.latin1, T.gawk.
254
255 Jan 5, 2013:
256 added ,NULL initializer to static Cells in run.c; not really
257 needed but cleaner. Thanks to Michael Bombardieri.
258
259 Dec 20, 2012:
260 fiddled makefile to get correct yacc and bison flags. pick yacc
261 (linux) or bison (mac) as necessary.
262
263 added __attribute__((__noreturn__)) to a couple of lines in
264 proto.h, to silence someone's enthusiastic checker.
265
266 fixed obscure call by value bug in split(a[1],a) reported on
267 9fans. the management of temporary values is just a mess; i
268 took a shortcut by making an extra string copy. thanks
269 to paul patience and arnold robbins for passing it on and for
270 proposed patches.
271
272 tiny fiddle in setfval to eliminate -0 results in T.expr, which
273 has irritated me for 20+ years.
274
275 Aug 10, 2011:
276 another fix to avoid core dump with delete(ARGV); again, many thanks
277 to ruslan ermilov.
278
279 Aug 7, 2011:
280 split(s, a, //) now behaves the same as split(s, a, "")
281
282 Jun 12, 2011:
283 /pat/, \n /pat/ {...} is now legal, though bad style to use.
284
285 added checks to new -v code that permits -vnospace; thanks to
286 ruslan ermilov for spotting this and providing the patch.
287
288 removed fixed limit on number of open files; thanks to aleksey
289 cheusov and christos zoulos.
290
291 fixed day 1 bug that resurrected deleted elements of ARGV when
292 used as filenames (in lib.c).
293
294 minor type fiddles to make gcc -Wall -pedantic happier (but not
295 totally so); turned on -fno-strict-aliasing in makefile.
296
297 May 6, 2011:
298 added #ifdef for isblank.
299 now allows -ffoo as well as -f foo arguments.
300 (thanks, ruslan)
301
302 May 1, 2011:
303 after advice from todd miller, kevin lo, ruslan ermilov,
304 and arnold robbins, changed srand() to return the previous
305 seed (which is 1 on the first call of srand). the seed is
306 an Awkfloat internally though converted to unsigned int to
307 pass to the library srand(). thanks, everyone.
308
309 fixed a subtle (and i hope low-probability) overflow error
310 in fldbld, by adding space for one extra \0. thanks to
311 robert bassett for spotting this one and providing a fix.
312
313 removed the files related to compilation on windows. i no
314 longer have anything like a current windows environment, so
315 i can't test any of it.
316
317 May 23, 2010:
318 fixed long-standing overflow bug in run.c; many thanks to
319 nelson beebe for spotting it and providing the fix.
320
321 fixed bug that didn't parse -vd=1 properly; thanks to santiago
322 vila for spotting it.
323
324 Feb 8, 2010:
325 i give up. replaced isblank with isspace in b.c; there are
326 no consistent header files.
327
328 Nov 26, 2009:
329 fixed a long-standing issue with when FS takes effect. a
330 change to FS is now noticed immediately for subsequent splits.
331
332 changed the name getline() to awkgetline() to avoid yet another
333 name conflict somewhere.
334
335 Feb 11, 2009:
336 temporarily for now defined HAS_ISBLANK, since that seems to
337 be the best way through the thicket. isblank arrived in C99,
338 but seems to be arriving at different systems at different
339 times.
340
341 Oct 8, 2008:
342 fixed typo in b.c that set tmpvec wrongly. no one had ever
343 run into the problem, apparently. thanks to alistair crooks.
344
345 Oct 23, 2007:
346 minor fix in lib.c: increase inputFS to 100, change malloc
347 for fields to n+1.
348
349 fixed memory fault caused by out of order test in setsval.
350
351 thanks to david o'brien, freebsd, for both fixes.
352
353 May 1, 2007:
354 fiddle in makefile to fix for BSD make; thanks to igor sobrado.
355
356 Mar 31, 2007:
357 fixed some null pointer refs calling adjbuf.
358
359 Feb 21, 2007:
360 fixed a bug in matching the null RE in sub and gsub. thanks to al aho
361 who actually did the fix (in b.c), and to wolfgang seeberg for finding
362 it and providing a very compact test case.
363
364 fixed quotation in b.c; thanks to Hal Pratt and the Princeton Dante
365 Project.
366
367 removed some no-effect asserts in run.c.
368
369 fiddled maketab.c to not complain about bison-generated values.
370
371 removed the obsolete -V argument; fixed --version to print the
372 version and exit.
373
374 fixed wording and an outright error in the usage message; thanks to igor
375 sobrado and jason mcintyre.
376
377 fixed a bug in -d that caused core dump if no program followed.
378
379 Jan 1, 2007:
380 dropped mac.code from makefile; there are few non-MacOSX
381 mac's these days.
382
383 Jan 17, 2006:
384 system() not flagged as unsafe in the unadvertised -safe option.
385 found it while enhancing tests before shipping the ;login: article.
386 practice what you preach.
387
388 removed the 9-years-obsolete -mr and -mf flags.
389
390 added -version and --version options.
391
392 core dump on linux with BEGIN {nextfile}, now fixed.
393
394 removed some #ifdef's in run.c and lex.c that appear to no
395 longer be necessary.
396
397 Apr 24, 2005:
398 modified lib.c so that values of $0 et al are preserved in the END
399 block, apparently as required by posix. thanks to havard eidnes
400 for the report and code.
401
402 Jan 14, 2005:
403 fixed infinite loop in parsing, originally found by brian tsang.
404 thanks to arnold robbins for a suggestion that started me
405 rethinking it.
406
407 Dec 31, 2004:
408 prevent overflow of -f array in main, head off potential error in
409 call of SYNTAX(), test malloc return in lib.c, all with thanks to
410 todd miller.
411
412 Dec 22, 2004:
413 cranked up size of NCHARS; coverity thinks it can be overrun with
414 smaller size, and i think that's right. added some assertions to b.c
415 to catch places where it might overrun. the RE code is still fragile.
416
417 Dec 5, 2004:
418 fixed a couple of overflow problems with ridiculous field numbers:
419 e.g., print $(2^32-1). thanks to ruslan ermilov, giorgos keramidas
420 and david o'brien at freebsd.org for patches. this really should
421 be re-done from scratch.
422
423 Nov 21, 2004:
424 fixed another 25-year-old RE bug, in split. it's another failure
425 to (re-)initialize. thanks to steve fisher for spotting this and
426 providing a good test case.
427
428 Nov 22, 2003:
429 fixed a bug in regular expressions that dates (so help me) from 1977;
430 it's been there from the beginning. an anchored longest match that
431 was longer than the number of states triggered a failure to initialize
432 the machine properly. many thanks to moinak ghosh for not only finding
433 this one but for providing a fix, in some of the most mysterious
434 code known to man.
435
436 fixed a storage leak in call() that appears to have been there since
437 1983 or so -- a function without an explicit return that assigns a
438 string to a parameter leaked a Cell. thanks to moinak ghosh for
439 spotting this very subtle one.
440
441 Jul 31, 2003:
442 fixed, thanks to andrey chernov and ruslan ermilov, a bug in lex.c
443 that mis-handled the character 255 in input. (it was being compared
444 to EOF with a signed comparison.)
445
446 Jul 29, 2003:
447 fixed (i think) the long-standing botch that included the beginning of
448 line state ^ for RE's in the set of valid characters; this led to a
449 variety of odd problems, including failure to properly match certain
450 regular expressions in non-US locales. thanks to ruslan for keeping
451 at this one.
452
453 Jul 28, 2003:
454 n-th try at getting internationalization right, with thanks to volker
455 kiefel, arnold robbins and ruslan ermilov for advice, though they
456 should not be blamed for the outcome. according to posix, "." is the
457 radix character in programs and command line arguments regardless of
458 the locale; otherwise, the locale should prevail for input and output
459 of numbers. so it's intended to work that way.
460
461 i have rescinded the attempt to use strcoll in expanding shorthands in
462 regular expressions (cclenter). its properties are much too
463 surprising; for example [a-c] matches aAbBc in locale en_US but abBcC
464 in locale fr_CA. i can see how this might arise by implementation
465 but i cannot explain it to a human user. (this behavior can be seen
466 in gawk as well; we're leaning on the same library.)
467
468 the issue appears to be that strcoll is meant for sorting, where
469 merging upper and lower case may make sense (though note that unix
470 sort does not do this by default either). it is not appropriate
471 for regular expressions, where the goal is to match specific
472 patterns of characters. in any case, the notations [:lower:], etc.,
473 are available in awk, and they are more likely to work correctly in
474 most locales.
475
476 a moratorium is hereby declared on internationalization changes.
477 i apologize to friends and colleagues in other parts of the world.
478 i would truly like to get this "right", but i don't know what
479 that is, and i do not want to keep making changes until it's clear.
480
481 Jul 4, 2003:
482 fixed bug that permitted non-terminated RE, as in "awk /x".
483
484 Jun 1, 2003:
485 subtle change to split: if source is empty, number of elems
486 is always 0 and the array is not set.
487
488 Mar 21, 2003:
489 added some parens to isblank, in another attempt to make things
490 internationally portable.
491
492 Mar 14, 2003:
493 the internationalization changes, somewhat modified, are now
494 reinstated. in theory awk will now do character comparisons
495 and case conversions in national language, but "." will always
496 be the decimal point separator on input and output regardless
497 of national language. isblank(){} has an #ifndef.
498
499 this no longer compiles on windows: LC_MESSAGES isn't defined
500 in vc6++.
501
502 fixed subtle behavior in field and record splitting: if FS is
503 a single character and RS is not empty, \n is NOT a separator.
504 this tortuous reading is found in the awk book; behavior now
505 matches gawk and mawk.
506
507 Dec 13, 2002:
508 for the moment, the internationalization changes of nov 29 are
509 rolled back -- programs like x = 1.2 don't work in some locales,
510 because the parser is expecting x = 1,2. until i understand this
511 better, this will have to wait.
512
513 Nov 29, 2002:
514 modified b.c (with tiny changes in main and run) to support
515 locales, using strcoll and iswhatever tests for posix character
516 classes. thanks to ruslan ermilov (ru (at) freebsd.org) for code.
517 the function isblank doesn't seem to have propagated to any
518 header file near me, so it's there explicitly. not properly
519 tested on non-ascii character sets by me.
520
521 Jun 28, 2002:
522 modified run/format() and tran/getsval() to do a slightly better
523 job on using OFMT for output from print and CONVFMT for other
524 number->string conversions, as promised by posix and done by
525 gawk and mawk. there are still places where it doesn't work
526 right if CONVFMT is changed; by then the STR attribute of the
527 variable has been irrevocably set. thanks to arnold robbins for
528 code and examples.
529
530 fixed subtle bug in format that could get core dump. thanks to
531 Jaromir Dolecek <jdolecek@NetBSD.org> for finding and fixing.
532 minor cleanup in run.c / format() at the same time.
533
534 added some tests for null pointers to debugging printf's, which
535 were never intended for external consumption. thanks to dave
536 kerns (dkerns (at) lucent.com) for pointing this out.
537
538 GNU compatibility: an empty regexp matches anything (thanks to
539 dag-erling smorgrav, des (at) ofug.org). subject to reversion if
540 this does more harm than good.
541
542 pervasive small changes to make things more const-correct, as
543 reported by gcc's -Wwrite-strings. as it says in the gcc manual,
544 this may be more nuisance than useful. provoked by a suggestion
545 and code from arnaud desitter, arnaud@nimbus.geog.ox.ac.uk
546
547 minor documentation changes to note that this now compiles out
548 of the box on Mac OS X.
549
550 Feb 10, 2002:
551 changed types in posix chars structure to quiet solaris cc.
552
553 Jan 1, 2002:
554 fflush() or fflush("") flushes all files and pipes.
555
556 length(arrayname) returns number of elements; thanks to
557 arnold robbins for suggestion.
558
559 added a makefile.win to make it easier to build on windows.
560 based on dan allen's buildwin.bat.
561
562 Nov 16, 2001:
563 added support for posix character class names like [:digit:],
564 which are not exactly shorter than [0-9] and perhaps no more
565 portable. thanks to dag-erling smorgrav for code.
566
567 Feb 16, 2001:
568 removed -m option; no longer needed, and it was actually
569 broken (noted thanks to volker kiefel).
570
571 Feb 10, 2001:
572 fixed an appalling bug in gettok: any sequence of digits, +,-, E, e,
573 and period was accepted as a valid number if it started with a period.
574 this would never have happened with the lex version.
575
576 other 1-character botches, now fixed, include a bare $ and a
577 bare " at the end of the input.
578
579 Feb 7, 2001:
580 more (const char *) casts in b.c and tran.c to silence warnings.
581
582 Nov 15, 2000:
583 fixed a bug introduced in august 1997 that caused expressions
584 like $f[1] to be syntax errors. thanks to arnold robbins for
585 noticing this and providing a fix.
586
587 Oct 30, 2000:
588 fixed some nextfile bugs: not handling all cases. thanks to
589 arnold robbins for pointing this out. new regressions added.
590
591 close() is now a function. it returns whatever the library
592 fclose returns, and -1 for closing a file or pipe that wasn't
593 opened.
594
595 Sep 24, 2000:
596 permit \n explicitly in character classes; won't work right
597 if comes in as "[\n]" but ok as /[\n]/, because of multiple
598 processing of \'s. thanks to arnold robbins.
599
600 July 5, 2000:
601 minor fiddles in tran.c to keep compilers happy about uschar.
602 thanks to norman wilson.
603
604 May 25, 2000:
605 yet another attempt at making 8-bit input work, with another
606 band-aid in b.c (member()), and some (uschar) casts to head
607 off potential errors in subscripts (like isdigit). also
608 changed HAT to NCHARS-2. thanks again to santiago vila.
609
610 changed maketab.c to ignore apparently out of range definitions
611 instead of halting; new freeBSD generates one. thanks to
612 jon snader <jsnader (at) ix.netcom.com> for pointing out the problem.
613
614 May 2, 2000:
615 fixed an 8-bit problem in b.c by making several char*'s into
616 unsigned char*'s. not clear i have them all yet. thanks to
617 Santiago Vila <sanvila (at) unex.es> for the bug report.
618
619 Apr 21, 2000:
620 finally found and fixed a memory leak in function call; it's
621 been there since functions were added ~1983. thanks to
622 jon bentley for the test case that found it.
623
624 added test in envinit to catch environment "variables" with
625 names beginning with '='; thanks to Berend Hasselman.
626
627 Jul 28, 1999:
628 added test in defn() to catch function foo(foo), which
629 otherwise recurses until core dump. thanks to arnold
630 robbins for noticing this.
631
632 Jun 20, 1999:
633 added *bp in gettok in lex.c; appears possible to exit function
634 without terminating the string. thanks to russ cox.
635
636 Jun 2, 1999:
637 added function stdinit() to run to initialize files[] array,
638 in case stdin, etc., are not constants; some compilers care.
639
640 May 10, 1999:
641 replaced the ERROR ... FATAL, etc., macros with functions
642 based on vprintf, to avoid problems caused by overrunning
643 fixed-size errbuf array. thanks to ralph corderoy for the
644 impetus, and for pointing out a string termination bug in
645 qstring as well.
646
647 Apr 21, 1999:
648 fixed bug that caused occasional core dumps with commandline
649 variable with value ending in \. (thanks to nelson beebe for
650 the test case.)
651
652 Apr 16, 1999:
653 with code kindly provided by Bruce Lilly, awk now parses
654 /=/ and similar constructs more sensibly in more places.
655 Bruce also provided some helpful test cases.
656
657 Apr 5, 1999:
658 changed true/false to True/False in run.c to make it
659 easier to compile with C++. Added some casts on malloc
660 and realloc to be honest about casts; ditto. changed
661 ltype int to long in struct rrow to reduce some 64-bit
662 complaints; other changes scattered throughout for the
663 same purpose. thanks to Nelson Beebe for these portability
664 improvements.
665
666 removed some horrible pointer-int casting in b.c and elsewhere
667 by adding ptoi and itonp to localize the casts, which are
668 all benign. fixed one incipient bug that showed up on sgi
669 in 64-bit mode.
670
671 reset lineno for new source file; include filename in error
672 message. also fixed line number error in continuation lines.
673 (thanks to Nelson Beebe for both of these.)
674
675 Mar 24, 1999:
676 Nelson Beebe notes that irix 5.3 yacc dies with a bogus
677 error; use a newer version or switch to bison, since sgi
678 is unlikely to fix it.
679
680 Mar 5, 1999:
681 changed isnumber to is_number to avoid the problem caused by
682 versions of ctype.h that include the name isnumber.
683
684 distribution now includes a script for building on a Mac,
685 thanks to Dan Allen.
686
687 Feb 20, 1999:
688 fixed memory leaks in run.c (call) and tran.c (setfval).
689 thanks to Stephen Nutt for finding these and providing the fixes.
690
691 Jan 13, 1999:
692 replaced srand argument by (unsigned int) in run.c;
693 avoids problem on Mac and potentially on Unix & Windows.
694 thanks to Dan Allen.
695
696 added a few (int) casts to silence useless compiler warnings.
697 e.g., errorflag= in run.c jump().
698
699 added proctab.c to the bundle outout; one less thing
700 to have to compile out of the box.
701
702 added calls to _popen and _pclose to the win95 stub for
703 pipes (thanks to Steve Adams for this helpful suggestion).
704 seems to work, though properties are not well understood
705 by me, and it appears that under some circumstances the
706 pipe output is truncated. Be careful.
707
708 Oct 19, 1998:
709 fixed a couple of bugs in getrec: could fail to update $0
710 after a getline var; because inputFS wasn't initialized,
711 could split $0 on every character, a misleading diversion.
712
713 fixed caching bug in makedfa: LRU was actually removing
714 least often used.
715
716 thanks to ross ridge for finding these, and for providing
717 great bug reports.
718
719 May 12, 1998:
720 fixed potential bug in readrec: might fail to update record
721 pointer after growing. thanks to dan levy for spotting this
722 and suggesting the fix.
723
724 Mar 12, 1998:
725 added -V to print version number and die.
726
727 [notify dave kerns, dkerns (at) dacsoup.ih.lucent.com]
728
729 Feb 11, 1998:
730 subtle silent bug in lex.c: if the program ended with a number
731 longer than 1 digit, part of the input would be pushed back and
732 parsed again because token buffer wasn't terminated right.
733 example: awk 'length($0) > 10'. blush. at least i found it
734 myself.
735
736 Aug 31, 1997:
737 s/adelete/awkdelete/: SGI uses this in malloc.h.
738 thanks to nelson beebe for pointing this one out.
739
740 Aug 21, 1997:
741 fixed some bugs in sub and gsub when replacement includes \\.
742 this is a dark, horrible corner, but at least now i believe that
743 the behavior is the same as gawk and the intended posix standard.
744 thanks to arnold robbins for advice here.
745
746 Aug 9, 1997:
747 somewhat regretfully, replaced the ancient lex-based lexical
748 analyzer with one written in C. it's longer, generates less code,
749 and more portable; the old one depended too much on mysterious
750 properties of lex that were not preserved in other environments.
751 in theory these recognize the same language.
752
753 now using strtod to test whether a string is a number, instead of
754 the convoluted original function. should be more portable and
755 reliable if strtod is implemented right.
756
757 removed now-pointless optimization in makefile that tries to avoid
758 recompilation when awkgram.y is changed but symbols are not.
759
760 removed most fixed-size arrays, though a handful remain, some
761 of which are unchecked. you have been warned.
762
763 Aug 4, 1997:
764 with some trepidation, replaced the ancient code that managed
765 fields and $0 in fixed-size arrays with arrays that grow on
766 demand. there is still some tension between trying to make this
767 run fast and making it clean; not sure it's right yet.
768
769 the ill-conceived -mr and -mf arguments are now useful only
770 for debugging. previous dynamic string code removed.
771
772 numerous other minor cleanups along the way.
773
774 Jul 30, 1997:
775 using code provided by dan levy (to whom profuse thanks), replaced
776 fixed-size arrays and awkward kludges by a fairly uniform mechanism
777 to grow arrays as needed for printf, sub, gsub, etc.
778
779 Jul 23, 1997:
780 falling off the end of a function returns "" and 0, not 0.
781 thanks to arnold robbins.
782
783 Jun 17, 1997:
784 replaced several fixed-size arrays by dynamically-created ones
785 in run.c; added overflow tests to some previously unchecked cases.
786 getline, toupper, tolower.
787
788 getline code is still broken in that recursive calls may wind
789 up using the same space. [fixed later]
790
791 increased RECSIZE to 8192 to push problems further over the horizon.
792
793 added \r to \n as input line separator for programs, not data.
794 damn CRLFs.
795
796 modified format() to permit explicit printf("%c", 0) to include
797 a null byte in output. thanks to ken stailey for the fix.
798
799 added a "-safe" argument that disables file output (print >,
800 print >>), process creation (cmd|getline, print |, system), and
801 access to the environment (ENVIRON). this is a first approximation
802 to a "safe" version of awk, but don't rely on it too much. thanks
803 to joan feigenbaum and matt blaze for the inspiration long ago.
804
805 Jul 8, 1996:
806 fixed long-standing bug in sub, gsub(/a/, "\\\\&"); thanks to
807 ralph corderoy.
808
809 Jun 29, 1996:
810 fixed awful bug in new field splitting; didn't get all the places
811 where input was done.
812
813 Jun 28, 1996:
814 changed field-splitting to conform to posix definition: fields are
815 split using the value of FS at the time of input; it used to be
816 the value when the field or NF was first referred to, a much less
817 predictable definition. thanks to arnold robbins for encouragement
818 to do the right thing.
819
820 May 28, 1996:
821 fixed appalling but apparently unimportant bug in parsing octal
822 numbers in reg exprs.
823
824 explicit hex in reg exprs now limited to 2 chars: \xa, \xaa.
825
826 May 27, 1996:
827 cleaned up some declarations so gcc -Wall is now almost silent.
828
829 makefile now includes backup copies of ytab.c and lexyy.c in case
830 one makes before looking; it also avoids recreating lexyy.c unless
831 really needed.
832
833 s/aprintf/awkprint, s/asprintf/awksprintf/ to avoid some name clashes
834 with unwisely-written header files.
835
836 thanks to jeffrey friedl for several of these.
837
838 May 26, 1996:
839 an attempt to rationalize the (unsigned) char issue. almost all
840 instances of unsigned char have been removed; the handful of places
841 in b.c where chars are used as table indices have been hand-crafted.
842 added some latin-1 tests to the regression, but i'm not confident;
843 none of my compilers seem to care much. thanks to nelson beebe for
844 pointing out some others that do care.
845
846 May 2, 1996:
847 removed all register declarations.
848
849 enhanced split(), as in gawk, etc: split(s, a, "") splits s into
850 a[1]...a[length(s)] with each character a single element.
851
852 made the same changes for field-splitting if FS is "".
853
854 added nextfile, as in gawk: causes immediate advance to next
855 input file. (thanks to arnold robbins for inspiration and code).
856
857 small fixes to regexpr code: can now handle []], [[], and
858 variants; [] is now a syntax error, rather than matching
859 everything; [z-a] is now empty, not z. far from complete
860 or correct, however. (thanks to jeffrey friedl for pointing out
861 some awful behaviors.)
862
863 Apr 29, 1996:
864 replaced uchar by uschar everywhere; apparently some compilers
865 usurp this name and this causes conflicts.
866
867 fixed call to time in run.c (bltin); arg is time_t *.
868
869 replaced horrible pointer/long punning in b.c by a legitimate
870 union. should be safer on 64-bit machines and cleaner everywhere.
871 (thanks to nelson beebe for pointing out some of these problems.)
872
873 replaced nested comments by #if 0...#endif in run.c, lib.c.
874
875 removed getsval, setsval, execute macros from run.c and lib.c.
876 machines are 100x faster than they were when these macros were
877 first used.
878
879 revised filenames: awk.g.y => awkgram.y, awk.lx.l => awklex.l,
880 y.tab.[ch] => ytab.[ch], lex.yy.c => lexyy.c, all in the aid of
881 portability to nameless systems.
882
883 "make bundle" now includes yacc and lex output files for recipients
884 who don't have yacc or lex.
885
886 Aug 15, 1995:
887 initialized Cells in setsymtab more carefully; some fields
888 were not set. (thanks to purify, all of whose complaints i
889 think i now understand.)
890
891 fixed at least one error in gsub that looked at -1-th element
892 of an array when substituting for a null match (e.g., $).
893
894 delete arrayname is now legal; it clears the elements but leaves
895 the array, which may not be the right behavior.
896
897 modified makefile: my current make can't cope with the test used
898 to avoid unnecessary yacc invocations.
899
900 Jul 17, 1995:
901 added dynamically growing strings to awk.lx.l and b.c
902 to permit regular expressions to be much bigger.
903 the state arrays can still overflow.
904
905 Aug 24, 1994:
906 detect duplicate arguments in function definitions (mdm).
907
908 May 11, 1994:
909 trivial fix to printf to limit string size in sub().
910
911 Apr 22, 1994:
912 fixed yet another subtle self-assignment problem:
913 $1 = $2; $1 = $1 clobbered $1.
914
915 Regression tests now use private echo, to avoid quoting problems.
916
917 Feb 2, 1994:
918 changed error() to print line number as %d, not %g.
919
920 Jul 23, 1993:
921 cosmetic changes: increased sizes of some arrays,
922 reworded some error messages.
923
924 added CONVFMT as in posix (just replaced OFMT in getsval)
925
926 FILENAME is now "" until the first thing that causes a file
927 to be opened.
928
929 Nov 28, 1992:
930 deleted yyunput and yyoutput from proto.h;
931 different versions of lex give these different declarations.
932
933 May 31, 1992:
934 added -mr N and -mf N options: more record and fields.
935 these really ought to adjust automatically.
936
937 cleaned up some error messages; "out of space" now means
938 malloc returned NULL in all cases.
939
940 changed rehash so that if it runs out, it just returns;
941 things will continue to run slow, but maybe a bit longer.
942
943 Apr 24, 1992:
944 remove redundant close of stdin when using -f -.
945
946 got rid of core dump with -d; awk -d just prints date.
947
948 Apr 12, 1992:
949 added explicit check for /dev/std(in,out,err) in redirection.
950 unlike gawk, no /dev/fd/n yet.
951
952 added (file/pipe) builtin. hard to test satisfactorily.
953 not posix.
954
955 Feb 20, 1992:
956 recompile after abortive changes; should be unchanged.
957
958 Dec 2, 1991:
959 die-casting time: converted to ansi C, installed that.
960
961 Nov 30, 1991:
962 fixed storage leak in freefa, failing to recover [N]CCL.
963 thanks to Bill Jones (jones (at) cs.usask.ca)
964
965 Nov 19, 1991:
966 use RAND_MAX instead of literal in builtin().
967
968 Nov 12, 1991:
969 cranked up some fixed-size arrays in b.c, and added a test for
970 overflow in penter. thanks to mark larsen.
971
972 Sep 24, 1991:
973 increased buffer in gsub. a very crude fix to a general problem.
974 and again on Sep 26.
975
976 Aug 18, 1991:
977 enforce variable name syntax for commandline variables: has to
978 start with letter or _.
979
980 Jul 27, 1991:
981 allow newline after ; in for statements.
982
983 Jul 21, 1991:
984 fixed so that in self-assignment like $1=$1, side effects
985 like recomputing $0 take place. (this is getting subtle.)
986
987 Jun 30, 1991:
988 better test for detecting too-long output record.
989
990 Jun 2, 1991:
991 better defense against very long printf strings.
992 made break and continue illegal outside of loops.
993
994 May 13, 1991:
995 removed extra arg on gettemp, tempfree. minor error message rewording.
996
997 May 6, 1991:
998 fixed silly bug in hex parsing in hexstr().
999 removed an apparently unnecessary test in isnumber().
1000 warn about weird printf conversions.
1001 fixed unchecked array overwrite in relex().
1002
1003 changed for (i in array) to access elements in sorted order.
1004 then unchanged it -- it really does run slower in too many cases.
1005 left the code in place, commented out.
1006
1007 Feb 10, 1991:
1008 check error status on all writes, to avoid banging on full disks.
1009
1010 Jan 28, 1991:
1011 awk -f - reads the program from stdin.
1012
1013 Jan 11, 1991:
1014 failed to set numeric state on $0 in cmd|getline context in run.c.
1015
1016 Nov 2, 1990:
1017 fixed sleazy test for integrality in getsval; use modf.
1018
1019 Oct 29, 1990:
1020 fixed sleazy buggy code in lib.c that looked (incorrectly) for
1021 too long input lines.
1022
1023 Oct 14, 1990:
1024 fixed the bug on p. 198 in which it couldn't deduce that an
1025 argument was an array in some contexts. replaced the error
1026 message in intest() by code that damn well makes it an array.
1027
1028 Oct 8, 1990:
1029 fixed horrible bug: types and values were not preserved in
1030 some kinds of self-assignment. (in assign().)
1031
1032 Aug 24, 1990:
1033 changed NCHARS to 256 to handle 8-bit characters in strings
1034 presented to match(), etc.
1035
1036 Jun 26, 1990:
1037 changed struct rrow (awk.h) to use long instead of int for lval,
1038 since cfoll() stores a pointer in it. now works better when int's
1039 are smaller than pointers!
1040
1041 May 6, 1990:
1042 AVA fixed the grammar so that ! is uniformly of the same precedence as
1043 unary + and -. This renders illegal some constructs like !x=y, which
1044 now has to be parenthesized as !(x=y), and makes others work properly:
1045 !x+y is (!x)+y, and x!y is x !y, not two pattern-action statements.
1046 (These problems were pointed out by Bob Lenk of Posix.)
1047
1048 Added \x to regular expressions (already in strings).
1049 Limited octal to octal digits; \8 and \9 are not octal.
1050 Centralized the code for parsing escapes in regular expressions.
1051 Added a bunch of tests to T.re and T.sub to verify some of this.
1052
1053 Feb 9, 1990:
1054 fixed null pointer dereference bug in main.c: -F[nothing]. sigh.
1055
1056 restored srand behavior: it returns the current seed.
1057
1058 Jan 18, 1990:
1059 srand now returns previous seed value (0 to start).
1060
1061 Jan 5, 1990:
1062 fix potential problem in tran.c -- something was freed,
1063 then used in freesymtab.
1064
1065 Oct 18, 1989:
1066 another try to get the max number of open files set with
1067 relatively machine-independent code.
1068
1069 small fix to input() in case of multiple reads after EOF.
1070
1071 Oct 11, 1989:
1072 FILENAME is now defined in the BEGIN block -- too many old
1073 programs broke.
1074
1075 "-" means stdin in getline as well as on the commandline.
1076
1077 added a bunch of casts to the code to tell the truth about
1078 char * vs. unsigned char *, a right royal pain. added a
1079 setlocale call to the front of main, though probably no one
1080 has it usefully implemented yet.
1081
1082 Aug 24, 1989:
1083 removed redundant relational tests against nullnode if parse
1084 tree already had a relational at that point.
1085
1086 Aug 11, 1989:
1087 fixed bug: commandline variable assignment has to look like
1088 var=something. (consider the man page for =, in file =.1)
1089
1090 changed number of arguments to functions to static arrays
1091 to avoid repeated malloc calls.
1092
1093 Aug 2, 1989:
1094 restored -F (space) separator
1095
1096 Jul 30, 1989:
1097 added -v x=1 y=2 ... for immediate commandline variable assignment;
1098 done before the BEGIN block for sure. they have to precede the
1099 program if the program is on the commandline.
1100 Modified Aug 2 to require a separate -v for each assignment.
1101
1102 Jul 10, 1989:
1103 fixed ref-thru-zero bug in environment code in tran.c
1104
1105 Jun 23, 1989:
1106 add newline to usage message.
1107
1108 Jun 14, 1989:
1109 added some missing ansi printf conversion letters: %i %X %E %G.
1110 no sensible meaning for h or L, so they may not do what one expects.
1111
1112 made %* conversions work.
1113
1114 changed x^y so that if n is a positive integer, it's done
1115 by explicit multiplication, thus achieving maximum accuracy.
1116 (this should be done by pow() but it seems not to be locally.)
1117 done to x ^= y as well.
1118
1119 Jun 4, 1989:
1120 ENVIRON array contains environment: if shell variable V=thing,
1121 ENVIRON["V"] is "thing"
1122
1123 multiple -f arguments permitted. error reporting is naive.
1124 (they were permitted before, but only the last was used.)
1125
1126 fixed a really stupid botch in the debugging macro dprintf
1127
1128 fixed order of evaluation of commandline assignments to match
1129 what the book claims: an argument of the form x=e is evaluated
1130 at the time it would have been opened if it were a filename (p 63).
1131 this invalidates the suggested answer to ex 4-1 (p 195).
1132
1133 removed some code that permitted -F (space) fieldseparator,
1134 since it didn't quite work right anyway. (restored aug 2)
1135
1136 Apr 27, 1989:
1137 Line number now accumulated correctly for comment lines.
1138
1139 Apr 26, 1989:
1140 Debugging output now includes a version date,
1141 if one compiles it into the source each time.
1142
1143 Apr 9, 1989:
1144 Changed grammar to prohibit constants as 3rd arg of sub and gsub;
1145 prevents class of overwriting-a-constant errors. (Last one?)
1146 This invalidates the "banana" example on page 43 of the book.
1147
1148 Added \a ("alert"), \v (vertical tab), \xhhh (hexadecimal),
1149 as in ANSI, for strings. Rescinded the sloppiness that permitted
1150 non-octal digits in \ooo. Warning: not all compilers and libraries
1151 will be able to deal with \x correctly.
1152
1153 Jan 9, 1989:
1154 Fixed bug that caused tempcell list to contain a duplicate.
1155 The fix is kludgy.
1156
1157 Dec 17, 1988:
1158 Catches some more commandline errors in main.
1159 Removed redundant decl of modf in run.c (confuses some compilers).
1160 Warning: there's no single declaration of malloc, etc., in awk.h
1161 that seems to satisfy all compilers.
1162
1163 Dec 7, 1988:
1164 Added a bit of code to error printing to avoid printing nulls.
1165 (Not clear that it actually would.)
1166
1167 Nov 27, 1988:
1168 With fear and trembling, modified the grammar to permit
1169 multiple pattern-action statements on one line without
1170 an explicit separator. By definition, this capitulation
1171 to the ghost of ancient implementations remains undefined
1172 and thus subject to change without notice or apology.
1173 DO NOT COUNT ON IT.
1174
1175 Oct 30, 1988:
1176 Fixed bug in call() that failed to recover storage.
1177
1178 A warning is now generated if there are more arguments
1179 in the call than in the definition (in lieu of fixing
1180 another storage leak).
1181
1182 Oct 20, 1988:
1183 Fixed %c: if expr is numeric, use numeric value;
1184 otherwise print 1st char of string value. still
1185 doesn't work if the value is 0 -- won't print \0.
1186
1187 Added a few more checks for running out of malloc.
1188
1189 Oct 12, 1988:
1190 Fixed bug in call() that freed local arrays twice.
1191
1192 Fixed to handle deletion of non-existent array right;
1193 complains about attempt to delete non-array element.
1194
1195 Sep 30, 1988:
1196 Now guarantees to evaluate all arguments of built-in
1197 functions, as in C; the appearance is that arguments
1198 are evaluated before the function is called. Places
1199 affected are sub (gsub was ok), substr, printf, and
1200 all the built-in arithmetic functions in bltin().
1201 A warning is generated if a bltin() is called with
1202 the wrong number of arguments.
1203
1204 This requires changing makeprof on p167 of the book.
1205
1206 Aug 23, 1988:
1207 setting FILENAME in BEGIN caused core dump, apparently
1208 because it was freeing space not allocated by malloc.
1209
1210 July 24, 1988:
1211 fixed egregious error in toupper/tolower functions.
1212 still subject to rescinding, however.
1213
1214 July 2, 1988:
1215 flush stdout before opening file or pipe
1216
1217 July 2, 1988:
1218 performance bug in b.c/cgoto(): not freeing some sets of states.
1219 partial fix only right now, and the number of states increased
1220 to make it less obvious.
1221
1222 June 1, 1988:
1223 check error status on close
1224
1225 May 28, 1988:
1226 srand returns seed value it's using.
1227 see 1/18/90
1228
1229 May 22, 1988:
1230 Removed limit on depth of function calls.
1231
1232 May 10, 1988:
1233 Fixed lib.c to permit _ in commandline variable names.
1234
1235 Mar 25, 1988:
1236 main.c fixed to recognize -- as terminator of command-
1237 line options. Illegal options flagged.
1238 Error reporting slightly cleaned up.
1239
1240 Dec 2, 1987:
1241 Newer C compilers apply a strict scope rule to extern
1242 declarations within functions. Two extern declarations in
1243 lib.c and tran.c have been moved to obviate this problem.
1244
1245 Oct xx, 1987:
1246 Reluctantly added toupper and tolower functions.
1247 Subject to rescinding without notice.
1248
1249 Sep 17, 1987:
1250 Error-message printer had printf(s) instead of
1251 printf("%s",s); got core dumps when the message
1252 included a %.
1253
1254 Sep 12, 1987:
1255 Very long printf strings caused core dump;
1256 fixed aprintf, asprintf, format to catch them.
1257 Can still get a core dump in printf itself.
1258
1259
1260