| History log of /src/usr.sbin/makemandb |
| Revision | Date | Author | Comments |
| 1.3 | 18-May-2019 |
leot | Document name_desc attribute of mandb_links.
Discussed with <abhinav> via PR misc/54213, thanks!
|
| 1.2 | 07-May-2012 |
wiz | branches: 1.2.32; PR 46419 by Abhinav Upadhyay using his updated patch: Clean up after removing man page aliases.
|
| 1.1 | 07-Feb-2012 |
joerg | branches: 1.1.2; 1.1.4; Import the new apropos/whatis.
This code has been developed by Abhinav Upadhyay as part of Google's Summer of Code 2011. It uses libmandoc to parse man pages and builds a Full Text Index in a SQLite database. The combination of indexing the full manual page, filtering out stop words and ranking individual matches based on the section gives a much improved user experience.
The old makewhatis and friends are kept under MKMAKEMANDB=no for now.
|
| 1.1.4.3 | 23-May-2012 |
yamt | sync with head.
|
| 1.1.4.2 | 17-Apr-2012 |
yamt | sync with head
|
| 1.1.4.1 | 07-Feb-2012 |
yamt | file DBSCHEMA was added on branch yamt-pagecache on 2012-04-17 00:09:49 +0000
|
| 1.1.2.1 | 09-May-2012 |
riz | Pull up following revision(s) (requested by wiz in ticket #229): usr.sbin/makemandb/makemandb.c: revision 1.9 usr.sbin/makemandb/DBSCHEMA: revision 1.2 usr.sbin/makemandb/apropos-utils.c: revision 1.5 usr.sbin/makemandb/apropos-utils.h: revision 1.3 PR 46419 by Abhinav Upadhyay using his updated patch: Clean up after removing man page aliases.
|
| 1.2.32.1 | 10-Jun-2019 |
christos | Sync with HEAD
|
| 1.13 | 13-Nov-2024 |
riastradh | makemandb(8): Fix static build with PROGDPLIBS.
XXX This shouldn't have to mention transitive dependencies -- we should have a mechanism for upstream build goo for libarchive to transmit the information automatically to downstream build goo for makemandb.
|
| 1.12 | 12-Nov-2020 |
nia | branches: 1.12.8; Revert addition of pthread dependency on sqlite.
It is less trivial than expected and introduced some surprising breakage.
|
| 1.11 | 09-Nov-2020 |
martin | Fix copy&pasto in previous
|
| 1.10 | 08-Nov-2020 |
nia | sqlite: do not build without multithreading support
at least a few pkgsrc packages avoid base sqlite because it fails this check, and it's probably a surprising performance penalty for unsuspecting users
|
| 1.9 | 18-Jun-2017 |
abhinav | Add a custom tokenizer which does not stem certain keywords.
Which keywords should not be stemmed is specified in the nostem.txt file. (Right now I have taken all the man page names, split them if they had underscores, removed common English words and converted everything to lowercase.)
The tokenizer itself is based on the Porter stemming tokenizer shipped with Sqlite. The code in custom_apropos_tokenizer.c is copy of that code with some modifications to prevent stemming keywords specified in nostem.txt.
Additionally, it now uses underscore `_' also as a token delimiter. Therefore, now it's possible to do query for `lwp' and all `_lwp_*' man page names will be matched. Or the query can be `unconst' and `__UNCONST' will be matched. This was not possible earlier, because underscore was not a delimiter and therefore the index would have __UNCONST as a key rather than UNCONST.
The tokenizer needs fts3_tokenizer.h file, which is not shipped with the amalgamation build of Sqlite, therefore it needs to be added here (unless we decide there is a better place for it).
To enforce using the new tokenizer, a schema version bump is needed
Since the tokenization is done both at the indexing time (via makemandb) and also while query time (via apropos or whatis), it will be needed to bump the schema version everytime nostem.txt is modified. Otherwise the index will consist of old tokens and desired changes will not be seen with apropos.
This should also fix the issue reported in PR bin/46255. Similar suggestion was also made on tech-userlevel@ recently: <http://mail-index.netbsd.org/tech-userlevel/2017/06/08/msg010620.html>
Thanks to christos@ for multiple rounds of reviews of the tokenizer code.
|
| 1.8 | 21-May-2017 |
riastradh | Remove MKCRYPTO option.
Originally, MKCRYPTO was introduced because the United States classified cryptography as a munition and restricted its export. The export controls were substantially relaxed fifteen years ago, and are essentially irrelevant for software with published source code.
In the intervening time, nobody bothered to remove the option after its motivation -- the US export restriction -- was eliminated. I'm not aware of any other operating system that has a similar option; I expect it is mainly out of apathy for churn that we still have it. Today, cryptography is an essential part of modern computing -- you can't use the internet responsibly without cryptography.
The position of the TNF board of directors is that TNF makes no representation that MKCRYPTO=no satisfies any country's cryptography regulations.
My personal position is that the availability of cryptography is a basic human right; that any local laws restricting it to a privileged few are fundamentally immoral; and that it is wrong for developers to spend effort crippling cryptography to work around such laws.
As proposed on tech-crypto, tech-security, and tech-userlevel to no objections:
https://mail-index.netbsd.org/tech-crypto/2017/05/06/msg000719.html https://mail-index.netbsd.org/tech-security/2017/05/06/msg000928.html https://mail-index.netbsd.org/tech-userlevel/2017/05/06/msg010547.html
P.S. Reviewing all the uses of MKCRYPTO in src revealed a lot of *bad* crypto that was conditional on it, e.g. DES in telnet... That should probably be removed too, but on the grounds that it is bad, not on the grounds that it is (nominally) crypto.
|
| 1.7 | 21-Apr-2017 |
christos | libarchive now needs crypto
|
| 1.6 | 21-Jul-2016 |
christos | Add -lz to makefile to fix the build.
|
| 1.5 | 14-Jan-2013 |
christos | branches: 1.5.12; Fix the #ifdef NOTYET, namely use the terminal standout mode to highlight the search terms.
|
| 1.4 | 14-Jan-2013 |
christos | Since mdocml decided to name headers that conflict with system ones (term.h) move the header inclusion one up.
|
| 1.3 | 06-Oct-2012 |
wiz | Make mandb path configurable. makemandb (and related tools) use the path from the _mandb variable from man.conf now.
Set _mandb in man.conf to same value as was used before.
From Abhinav Upadhyay <er.abhinav.upadhyay@gmail.com>.
|
| 1.2 | 16-Feb-2012 |
joerg | branches: 1.2.2; 1.2.4; Add support for compressed man pages in all the usual formats.
|
| 1.1 | 07-Feb-2012 |
joerg | branches: 1.1.2; Import the new apropos/whatis.
This code has been developed by Abhinav Upadhyay as part of Google's Summer of Code 2011. It uses libmandoc to parse man pages and builds a Full Text Index in a SQLite database. The combination of indexing the full manual page, filtering out stop words and ranking individual matches based on the section gives a much improved user experience.
The old makewhatis and friends are kept under MKMAKEMANDB=no for now.
|
| 1.1.2.1 | 18-Feb-2012 |
riz | Pull up following revision(s) (requested by joerg in ticket #4): usr.sbin/makemandb/apropos.c: revision 1.5 external/bsd/mdocml/dist/read.c: revision 1.6 external/bsd/mdocml/dist/mandoc.h: revision 1.2 usr.sbin/makemandb/makemandb.c: revision 1.3 usr.sbin/makemandb/makemandb.c: revision 1.4 usr.sbin/makemandb/makemandb.c: revision 1.5 usr.sbin/makemandb/Makefile: revision 1.2 etc/man.conf: revision 1.30 usr.sbin/makemandb/apropos.1: revision 1.3 external/bsd/mdocml/Makefile.inc: revision 1.15 Be a bit more friendly to man pages using the roff .so command by changing the current directory to the parent of the man -p entry, e.g. /usr/share/man for /usr/share/man1. Also handle hyphen replacement if it was used as plain input and no backslash sequence was used at all in the line. Add support for apropos -s like in the old apropos. Fix capitalisation. Add note about optional quotation. From Abhinav Upadhyay. Merge mandoc.h r1.99 and read.c 1.28 from upstream: Add mparse_readmem, which allows application to run the parser directly from memory, e.g. after de-compressing a document. Add support for compressed man pages in all the usual formats. Use mmap when dealing with plain files. Add entries for bzip2 and xz.
|
| 1.2.4.2 | 25-Feb-2013 |
tls | resync with head
|
| 1.2.4.1 | 20-Nov-2012 |
tls | Resync to 2012-11-19 00:00:00 UTC
|
| 1.2.2.4 | 23-Jan-2013 |
yamt | sync with head
|
| 1.2.2.3 | 30-Oct-2012 |
yamt | sync with head
|
| 1.2.2.2 | 17-Apr-2012 |
yamt | sync with head
|
| 1.2.2.1 | 16-Feb-2012 |
yamt | file Makefile was added on branch yamt-pagecache on 2012-04-17 00:09:49 +0000
|
| 1.5.12.2 | 26-Apr-2017 |
pgoyette | Sync with HEAD
|
| 1.5.12.1 | 26-Jul-2016 |
pgoyette | Sync with HEAD
|
| 1.12.8.1 | 02-Aug-2025 |
perseant | Sync with HEAD
|
| 1.5 | 11-Sep-2022 |
gutteridge | makemandb/*: fix spelling of database and consistency of SQLite
|
| 1.4 | 24-May-2016 |
abhinav | Bring man pages in sync with reality.
Remove man pages run_query_html.3 and run_query_pager.3 as the corresponding functions have been removed from apropos-utils.c Ok by wiz@
|
| 1.3 | 02-Apr-2013 |
wiz | Bump date.
|
| 1.2 | 02-Apr-2013 |
christos | instead of having a format and no format flag, and exposing various formatters, provide a format enum and expose html formatting too.
|
| 1.1 | 07-Feb-2012 |
joerg | branches: 1.1.4; 1.1.8; Import the new apropos/whatis.
This code has been developed by Abhinav Upadhyay as part of Google's Summer of Code 2011. It uses libmandoc to parse man pages and builds a Full Text Index in a SQLite database. The combination of indexing the full manual page, filtering out stop words and ranking individual matches based on the section gives a much improved user experience.
The old makewhatis and friends are kept under MKMAKEMANDB=no for now.
|
| 1.1.8.1 | 23-Jun-2013 |
tls | resync from head
|
| 1.1.4.3 | 22-May-2014 |
yamt | sync with head.
for a reference, the tree before this commit was tagged as yamt-pagecache-tag8.
this commit was splitted into small chunks to avoid a limitation of cvs. ("Protocol error: too many arguments")
|
| 1.1.4.2 | 17-Apr-2012 |
yamt | sync with head
|
| 1.1.4.1 | 07-Feb-2012 |
yamt | file apropos-utils.3 was added on branch yamt-pagecache on 2012-04-17 00:09:49 +0000
|
| 1.51 | 03-Aug-2023 |
rin | makemandb: trailing whitespace
|
| 1.50 | 11-Sep-2022 |
gutteridge | makemandb/*: fix spelling of database and consistency of SQLite
|
| 1.49 | 19-May-2022 |
gutteridge | apropos(1): improve error handling in edge cases
Patch from RVP on NetBSD-Users, with an additional comment tweak by me. Summary from RVP:
1. Ignore SIGPIPE so that we're not killed in the middle of some DB operation by a botched $PAGER:
$ env PAGER=/non-existent apropos -p ...
2. Return proper exit status in case of write errors:
$ apropos ... >/dev/full || echo fail
|
| 1.48 | 27-Nov-2021 |
rillig | usr.sbin: remove unnecessary CONSTCOND, lint no longer needs it
Since 2021-01-31, lint no longer requires a CONSTCOND comment in a do-while-0 statement since this is a common code pattern, especially in statement-like macros.
sed -i -E 's,} while \(/\* ?CONSTCOND ?\*/ ?0\),} while (0),' */*.[ch]
|
| 1.47 | 18-Aug-2019 |
abhinav | PR bin/54343: We want the callback_args.machine to be NULL if it is not present in the DB.
The previous commit fixed the problem of allowing apropos to not crash and produce output even if the database is missing values for certain mandatory fields, such as name, section etc. Normally we don't expect those values to be missing in the database but in case of parsing errors it can happen.
However, the machine architecture is an optional field since not all man pages are hardware specific so that should be allowed to be set to NULL if not present in the database.
|
| 1.46 | 15-Aug-2019 |
christos | PR/54343: Prevent NULL pointers in callback strings; use "*?*" for now to identify them.
|
| 1.45 | 07-Jun-2019 |
leot | branches: 1.45.2; Properly free section_clause.
|
| 1.44 | 18-May-2019 |
abhinav | PR misc/54213: Fix performance of whatis(1) when no matches are found
In revision 1.6 of whatis.c the query was modified to return matches for names found in MLINKS of the man pages as well. However it was slow. The reason probably being that it required a join. But more importantly the where condition on an FTS virtual table column is very slow. To avoid the join and the expensive where condition on the virtual table, add the name_desc column to the mandb_links table as well. This improves the performance of whatis(1) to the original level at the expense of slight data duplication.
Bump the schema to force database rebuild to take account for the new column addition
|
| 1.43 | 19-Apr-2019 |
abhinav | Memory allocated by sqlite3_mprintf should be free'd by sqlite3_free
This was causing memory corruption thus making apropos(1) fail in some cases. Specifically following options were broken and should be fixed with this commit:
-n option was causing a core dump apropos was giving warning when using -l and any of the section numbers as options as reported by paulg on current-users.
|
| 1.42 | 14-Apr-2019 |
abhinav | Set the snippet_length field of the callback_args
Because of this field not being set, apropos was failing to show snippet when piped to a pager or when used with -p argument.
|
| 1.41 | 07-Mar-2019 |
christos | fix memory allocation problems detected by jemalloc...
|
| 1.40 | 25-Nov-2017 |
abhinav | branches: 1.40.4; Encapsulate all the arguments required by the query callback function in a struct.
If we want to add or remove arguments from the callback functions, it requires changing the callback interface all over the place. By letting the callback simply expect a single struct argument, it would clean things up a bit.
ok christos
|
| 1.39 | 01-Aug-2017 |
abhinav | Don't use the custom tokenizer when compiled with debugging on
Using the custom tokenizer means one cannot interactively query the database through the SQLite shell, thus thwarting the purpose of the debug build option.
Thanks to leot@ for reporting it.
(While there change the debug macro from DEBUG to APROPOS_DEBUG)
|
| 1.38 | 18-Jun-2017 |
abhinav | Add a custom tokenizer which does not stem certain keywords.
Which keywords should not be stemmed is specified in the nostem.txt file. (Right now I have taken all the man page names, split them if they had underscores, removed common English words and converted everything to lowercase.)
The tokenizer itself is based on the Porter stemming tokenizer shipped with Sqlite. The code in custom_apropos_tokenizer.c is copy of that code with some modifications to prevent stemming keywords specified in nostem.txt.
Additionally, it now uses underscore `_' also as a token delimiter. Therefore, now it's possible to do query for `lwp' and all `_lwp_*' man page names will be matched. Or the query can be `unconst' and `__UNCONST' will be matched. This was not possible earlier, because underscore was not a delimiter and therefore the index would have __UNCONST as a key rather than UNCONST.
The tokenizer needs fts3_tokenizer.h file, which is not shipped with the amalgamation build of Sqlite, therefore it needs to be added here (unless we decide there is a better place for it).
To enforce using the new tokenizer, a schema version bump is needed
Since the tokenization is done both at the indexing time (via makemandb) and also while query time (via apropos or whatis), it will be needed to bump the schema version everytime nostem.txt is modified. Otherwise the index will consist of old tokens and desired changes will not be seen with apropos.
This should also fix the issue reported in PR bin/46255. Similar suggestion was also made on tech-userlevel@ recently: <http://mail-index.netbsd.org/tech-userlevel/2017/06/08/msg010620.html>
Thanks to christos@ for multiple rounds of reviews of the tokenizer code.
|
| 1.37 | 01-May-2017 |
abhinav | Simplify handling of the section arguments in apropos(1).
Earlier, a white space separated string was generated containing all the section numbers passed through command line arguments. Later on that would have to be tokenized and processed. Instead of that, use a NULL terminated array of strings.
Thanks to christos@ for reviewing and suggesting further improvements.
|
| 1.36 | 30-Apr-2017 |
abhinav | Simplify
|
| 1.35 | 30-Apr-2017 |
abhinav | Instead of dereferencing the pointer passed in as function argument, use a temporary local buffer. Saves the cost of pointer dereferencing at so many places.
|
| 1.34 | 30-Apr-2017 |
abhinav | Update the comment to be in sync with the code.
|
| 1.33 | 30-Apr-2017 |
abhinav | Use sqlite3_mprintf() to generate SQL query instead of asprintf.
|
| 1.32 | 27-Apr-2017 |
abhinav | Disable the database compression if DEBUG is defined.
When debugging makemandb(8), it helps to be able to view the text being stored in the database.
|
| 1.31 | 23-Apr-2017 |
abhinav | branches: 1.31.2; Better handle MLINKS in apropos(1).
apropos(1) only indexes the first .Nm entry from the NAME section in the full text index. Rest of the .Nm entries are stored in a separate table: mandb_links.
Till now apropos(1) did not use the mandb_links table. So whenever a query was being made for one of the man page links, such as realloc(3), it was showing malloc(3) in the results but not as the first result. And, also the result would show up as malloc(3), rather than realloc(3) (which can be confusing).
With this change, for single keyword queries, apropos(1) would now utilise the mandb_links table as well. If the query is for one of the links of a man page, it would show as the first result. Also, the result would show up as the name of the link rather than the original man page name. For example, if the query was for realloc, the output would be realloc(3), rather than malloc(3).
Following are some example queries showing difference in the output before this change and after this change:
#Before changes $ apropos -n 5 -M realloc reallocarr (3) reallocate array reallocarray (3) reallocate memory for an array of elements checking for overflow fgetwln (3) get a line of wide characters from a stream fgetln (3) get a line from a stream posix_memalign (3) aligned memory allocation
#After changes $ ./apropos -n 5 -M realloc realloc (3) general memory allocation operations realloc (3) general purpose memory allocation functions realloc (9) general-purpose kernel memory allocator reallocarr (3) reallocate array reallocarray (3) reallocate memory for an array of elements checking for overflow
#Before changes $ apropos -n 5 -M TAILQ_REMOVE SLIST_HEAD (3) implementations of singly-linked lists, lists, simple queues, tail queues, and singly-linked tail queues
#After changes $ ./apropos -n 5 -M TAILQ_REMOVE TAILQ_REMOVE (3) implementations of singly-linked lists, lists, simple queues, tail queues, and singly-linked tail queues
#Before changes $ apropos -n 5 -M falloc filedesc (9) file descriptor tables and operations file (9) operations on file entries
#After changes $ ./apropos -n 5 -M falloc falloc (9) file descriptor tables and operations file (9) operations on file entries
ok christos@
|
| 1.30 | 10-Jan-2017 |
kamil | Include <unistd.h> for R_OK W_OK STDOUT_FILENO access(2)
These symbols are undefined after switch to new zlib.
|
| 1.29 | 03-Oct-2016 |
abhinav | Mark the section and md5_hash columns as unindexed in the FTS table, as they are not used for search
|
| 1.28 | 06-Jul-2016 |
abhinav | branches: 1.28.2; Fix an off by one issue when concatenating strings.
|
| 1.27 | 06-Jul-2016 |
abhinav | Fix possible buffer overflow when concatenating strings. Patch from christos@
|
| 1.26 | 01-Jun-2016 |
abhinav | Refactor the function for executing the search SQL query into two parts.
One part is responsible for generating the SQL query The other part is responsible for executing the generated query.
While there, also remove a comment which is not valid anymore. And, don't call the snippet function when doing legacy mode search as we are not using the full text feature there.
|
| 1.25 | 24-Apr-2016 |
christos | CID 1358675: Wrong variable test
|
| 1.24 | 13-Apr-2016 |
christos | PR/51062: Abhinav Upadhyay: Allow non numeric sections to be indexed and searched by apropos(1). Fold long lines.
|
| 1.23 | 13-Apr-2016 |
christos | PR/51038: Abhinav Upadhyay: check for access permissions to the sqlite database
|
| 1.22 | 31-Mar-2016 |
christos | PR/51025: Abhinav Upadhyay: Remove unused includes from apropos-utils.c
|
| 1.21 | 24-Mar-2016 |
christos | PR/51004: Abhinav Upadhyay: apropos html mode doesn't handle especial characters in the short description
|
| 1.20 | 20-Mar-2016 |
christos | PR/50460: Abhinav Upadhyay: Fix legacy apropos query to match both the name and the one line description and delete extra args.
|
| 1.19 | 03-Dec-2015 |
christos | CID 1341551: Don't bother formatting if ti == NULL
|
| 1.18 | 23-Nov-2015 |
christos | PR/50344: Stephen Fisher: apropos shows formatting on console with vt100 term type. Can't print terminfo sequences directly; need to process them with ti_puts() to handle padding. This removes the padding delays, and stricly could break on slow terminal hardware, but they way the code is structured makes it impossible to fix properly (since the formatting strings are passed in the query). XXX: pullup-7
|
| 1.17 | 18-Oct-2014 |
snj | src is too big these days to tolerate superfluous apostrophes. It's "its", people!
|
| 1.16 | 01-Aug-2014 |
wiz | branches: 1.16.2; Fix an off by one bug in apropos. The bug is in the html output where some garbage characters are seen in the context match output.
From Abhinav Upadhyay in PR 49058.
|
| 1.15 | 02-Apr-2013 |
christos | branches: 1.15.4; instead of having a format and no format flag, and exposing various formatters, provide a format enum and expose html formatting too.
|
| 1.14 | 29-Mar-2013 |
christos | fix legacy mode in pager filter. (don't ul format if we are not formatting).
|
| 1.13 | 29-Mar-2013 |
christos | - Fix legacy mode to use like instead of match. This loses ranking. - default to unlimited lines - fix formatting of legacy mode
|
| 1.12 | 29-Mar-2013 |
christos | - If the stdout is not a tty, prevent formatting unless forced with -i - Don't ever page unless asked for with -p - Introduce "legacy mode" (-l) 1. searches only name and name_desc, prints name(section) - name_description 2. turns off escape formatting (can be forced on with -i) 3. turns off context printing (can be forced on with -c) - Parse the environment $APROPOS variable as an argument vector.
With these changes one can simply 'export APROPOS=-l' and get the old apropos behavior.
|
| 1.11 | 10-Feb-2013 |
christos | add -r flag to elide tty formatting
|
| 1.10 | 10-Feb-2013 |
christos | remove trailing whitespace
|
| 1.9 | 14-Jan-2013 |
christos | - move the terminal handling in apropos-utils.c since htmp and pager are also handled there. - underline the name, section, and description so that it is prettier. - change to bold terminal the terminal highlighting to match with less
|
| 1.8 | 14-Jan-2013 |
christos | Since mdocml decided to name headers that conflict with system ones (term.h) move the header inclusion one up.
|
| 1.7 | 06-Oct-2012 |
wiz | Make mandb path configurable. makemandb (and related tools) use the path from the _mandb variable from man.conf now.
Set _mandb in man.conf to same value as was used before.
From Abhinav Upadhyay <er.abhinav.upadhyay@gmail.com>.
|
| 1.6 | 10-May-2012 |
joerg | branches: 1.6.2; KNF
|
| 1.5 | 07-May-2012 |
wiz | PR 46419 by Abhinav Upadhyay using his updated patch: Clean up after removing man page aliases.
|
| 1.4 | 15-Apr-2012 |
wiz | branches: 1.4.2; Handle pages with slashes in their names better. From Abhinav Upadhyay in private mail.
|
| 1.3 | 07-Apr-2012 |
apb | Add the result from sqlite3_errmsg() to some error messages. Now we can get "apropos: Unable to query schema version: database is locked" instead of just "apropos: Unable to query schema version".
|
| 1.2 | 07-Feb-2012 |
joerg | branches: 1.2.2; Fix C&P error with $NetBSD$
|
| 1.1 | 07-Feb-2012 |
joerg | Import the new apropos/whatis.
This code has been developed by Abhinav Upadhyay as part of Google's Summer of Code 2011. It uses libmandoc to parse man pages and builds a Full Text Index in a SQLite database. The combination of indexing the full manual page, filtering out stop words and ranking individual matches based on the section gives a much improved user experience.
The old makewhatis and friends are kept under MKMAKEMANDB=no for now.
|
| 1.2.2.2 | 09-May-2012 |
riz | Pull up following revision(s) (requested by wiz in ticket #229): usr.sbin/makemandb/makemandb.c: revision 1.9 usr.sbin/makemandb/DBSCHEMA: revision 1.2 usr.sbin/makemandb/apropos-utils.c: revision 1.5 usr.sbin/makemandb/apropos-utils.h: revision 1.3 PR 46419 by Abhinav Upadhyay using his updated patch: Clean up after removing man page aliases.
|
| 1.2.2.1 | 19-Apr-2012 |
riz | Pull up following revision(s) (requested by wiz in ticket #186): usr.sbin/makemandb/apropos.c: revision 1.6 usr.sbin/makemandb/apropos-utils.c: revision 1.3 usr.sbin/makemandb/apropos-utils.c: revision 1.4 Add the result from sqlite3_errmsg() to some error messages. Now we can get "apropos: Unable to query schema version: database is locked" instead of just "apropos: Unable to query schema version". Handle pages with slashes in their names better. From Abhinav Upadhyay in private mail.
|
| 1.4.2.6 | 22-May-2014 |
yamt | sync with head.
for a reference, the tree before this commit was tagged as yamt-pagecache-tag8.
this commit was splitted into small chunks to avoid a limitation of cvs. ("Protocol error: too many arguments")
|
| 1.4.2.5 | 23-Jan-2013 |
yamt | sync with head
|
| 1.4.2.4 | 30-Oct-2012 |
yamt | sync with head
|
| 1.4.2.3 | 23-May-2012 |
yamt | sync with head.
|
| 1.4.2.2 | 17-Apr-2012 |
yamt | sync with head
|
| 1.4.2.1 | 15-Apr-2012 |
yamt | file apropos-utils.c was added on branch yamt-pagecache on 2012-04-17 00:09:49 +0000
|
| 1.6.2.4 | 20-Aug-2014 |
tls | Rebase to HEAD as of a few days ago.
|
| 1.6.2.3 | 23-Jun-2013 |
tls | resync from head
|
| 1.6.2.2 | 25-Feb-2013 |
tls | resync with head
|
| 1.6.2.1 | 20-Nov-2012 |
tls | Resync to 2012-11-19 00:00:00 UTC
|
| 1.15.4.1 | 10-Aug-2014 |
tls | Rebase.
|
| 1.16.2.1 | 15-Apr-2016 |
snj | Pull up following revision(s) (requested by christos in ticket #1142): usr.sbin/makemandb/apropos-utils.c: revisions 1.18, 1.19 CID 1341551: Don't bother formatting if ti == NULL -- PR/50344: Stephen Fisher: apropos shows formatting on console with vt100 term type. Can't print terminfo sequences directly; need to process them with ti_puts() to handle padding. This removes the padding delays, and stricly could break on slow terminal hardware, but they way the code is structured makes it impossible to fix properly (since the formatting strings are passed in the query).
|
| 1.28.2.3 | 26-Apr-2017 |
pgoyette | Sync with HEAD
|
| 1.28.2.2 | 20-Mar-2017 |
pgoyette | Sync with HEAD
|
| 1.28.2.1 | 04-Nov-2016 |
pgoyette | Sync with HEAD
|
| 1.31.2.1 | 02-May-2017 |
pgoyette | Sync with HEAD - tag prg-localcount2-base1
|
| 1.40.4.2 | 13-Apr-2020 |
martin | Mostly merge changes from HEAD upto 20200411
|
| 1.40.4.1 | 10-Jun-2019 |
christos | Sync with HEAD
|
| 1.45.2.1 | 03-Jun-2022 |
martin | Pull up following revision(s) (requested by gutteridge in ticket #1461):
usr.sbin/makemandb/apropos.1: revision 1.19 usr.sbin/makemandb/apropos.c: revision 1.25 usr.sbin/makemandb/apropos.c: revision 1.26 usr.sbin/makemandb/apropos.1: revision 1.20 usr.sbin/makemandb/apropos.1: revision 1.21 usr.sbin/makemandb/apropos.1: revision 1.22 usr.sbin/makemandb/apropos.1: revision 1.23 usr.sbin/makemandb/apropos-utils.c: revision 1.46 usr.sbin/makemandb/apropos-utils.c: revision 1.47 usr.sbin/makemandb/apropos-utils.c: revision 1.49
PR/54343: Prevent NULL pointers in callback strings; use "*?*" for now to identify them.
PR bin/54343: We want the callback_args.machine to be NULL if it is not present in the DB.
The previous commit fixed the problem of allowing apropos to not crash and produce output even if the database is missing values for certain mandatory fields, such as name, section etc. Normally we don't expect those values to be missing in the database but in case of parsing errors it can happen.
However, the machine architecture is an optional field since not all man pages are hardware specific so that should be allowed to be set to NULL if not present in the database.
apropos.c: fix pager functionality
Issue reported by Rocky Hotas on NetBSD-Users, patch input from RVP on same, adjustments by me.
apropos.1: document the PAGER environment variable
apropos(1): use proper -width
apropos(1): use proper -width for the list of options too
apropos(1): Tweak the description of -1, ... -9, and -s
-s is not for compatibility only, because section names can be anything. E.g. we have 3lua and 9lua in base. We have rudiments of 3f (for FORTRAN libs). Some packages in pkgsrc also use suffixed 1 and 3 sections.
apropos(1): Use the official spelling for "SQLite". While here, use .Bx to refer to 3BSD.
apropos(1): improve error handling in edge cases Patch from RVP on NetBSD-Users, with an additional comment tweak by me.
Summary from RVP: 1. Ignore SIGPIPE so that we're not killed in the middle of some DB operation by a botched $PAGER: $ env PAGER=3D/non-existent apropos -p ... 2. Return proper exit status in case of write errors: $ apropos ... >/dev/full || echo fail
|
| 1.15 | 18-May-2019 |
abhinav | PR misc/54213: Fix performance of whatis(1) when no matches are found
In revision 1.6 of whatis.c the query was modified to return matches for names found in MLINKS of the man pages as well. However it was slow. The reason probably being that it required a join. But more importantly the where condition on an FTS virtual table column is very slow. To avoid the join and the expensive where condition on the virtual table, add the name_desc column to the mandb_links table as well. This improves the performance of whatis(1) to the original level at the expense of slight data duplication.
Bump the schema to force database rebuild to take account for the new column addition
|
| 1.14 | 25-Nov-2017 |
abhinav | branches: 1.14.4; Encapsulate all the arguments required by the query callback function in a struct.
If we want to add or remove arguments from the callback functions, it requires changing the callback interface all over the place. By letting the callback simply expect a single struct argument, it would clean things up a bit.
ok christos
|
| 1.13 | 18-Jun-2017 |
abhinav | Add a custom tokenizer which does not stem certain keywords.
Which keywords should not be stemmed is specified in the nostem.txt file. (Right now I have taken all the man page names, split them if they had underscores, removed common English words and converted everything to lowercase.)
The tokenizer itself is based on the Porter stemming tokenizer shipped with Sqlite. The code in custom_apropos_tokenizer.c is copy of that code with some modifications to prevent stemming keywords specified in nostem.txt.
Additionally, it now uses underscore `_' also as a token delimiter. Therefore, now it's possible to do query for `lwp' and all `_lwp_*' man page names will be matched. Or the query can be `unconst' and `__UNCONST' will be matched. This was not possible earlier, because underscore was not a delimiter and therefore the index would have __UNCONST as a key rather than UNCONST.
The tokenizer needs fts3_tokenizer.h file, which is not shipped with the amalgamation build of Sqlite, therefore it needs to be added here (unless we decide there is a better place for it).
To enforce using the new tokenizer, a schema version bump is needed
Since the tokenization is done both at the indexing time (via makemandb) and also while query time (via apropos or whatis), it will be needed to bump the schema version everytime nostem.txt is modified. Otherwise the index will consist of old tokens and desired changes will not be seen with apropos.
This should also fix the issue reported in PR bin/46255. Similar suggestion was also made on tech-userlevel@ recently: <http://mail-index.netbsd.org/tech-userlevel/2017/06/08/msg010620.html>
Thanks to christos@ for multiple rounds of reviews of the tokenizer code.
|
| 1.12 | 01-May-2017 |
abhinav | Simplify handling of the section arguments in apropos(1).
Earlier, a white space separated string was generated containing all the section numbers passed through command line arguments. Later on that would have to be tokenized and processed. Instead of that, use a NULL terminated array of strings.
Thanks to christos@ for reviewing and suggesting further improvements.
|
| 1.11 | 13-Apr-2016 |
christos | branches: 1.11.6; PR/51062: Abhinav Upadhyay: Allow non numeric sections to be indexed and searched by apropos(1). Fold long lines.
|
| 1.10 | 13-Apr-2016 |
christos | PR/51038: Abhinav Upadhyay: check for access permissions to the sqlite database
|
| 1.9 | 02-Apr-2013 |
christos | instead of having a format and no format flag, and exposing various formatters, provide a format enum and expose html formatting too.
|
| 1.8 | 29-Mar-2013 |
christos | - If the stdout is not a tty, prevent formatting unless forced with -i - Don't ever page unless asked for with -p - Introduce "legacy mode" (-l) 1. searches only name and name_desc, prints name(section) - name_description 2. turns off escape formatting (can be forced on with -i) 3. turns off context printing (can be forced on with -c) - Parse the environment $APROPOS variable as an argument vector.
With these changes one can simply 'export APROPOS=-l' and get the old apropos behavior.
|
| 1.7 | 10-Feb-2013 |
christos | add -r flag to elide tty formatting
|
| 1.6 | 10-Feb-2013 |
christos | remove trailing whitespace
|
| 1.5 | 14-Jan-2013 |
christos | - move the terminal handling in apropos-utils.c since htmp and pager are also handled there. - underline the name, section, and description so that it is prettier. - change to bold terminal the terminal highlighting to match with less
|
| 1.4 | 06-Oct-2012 |
wiz | Make mandb path configurable. makemandb (and related tools) use the path from the _mandb variable from man.conf now.
Set _mandb in man.conf to same value as was used before.
From Abhinav Upadhyay <er.abhinav.upadhyay@gmail.com>.
|
| 1.3 | 07-May-2012 |
wiz | branches: 1.3.2; PR 46419 by Abhinav Upadhyay using his updated patch: Clean up after removing man page aliases.
|
| 1.2 | 07-Feb-2012 |
joerg | branches: 1.2.2; 1.2.4; Fix C&P error with $NetBSD$
|
| 1.1 | 07-Feb-2012 |
joerg | Import the new apropos/whatis.
This code has been developed by Abhinav Upadhyay as part of Google's Summer of Code 2011. It uses libmandoc to parse man pages and builds a Full Text Index in a SQLite database. The combination of indexing the full manual page, filtering out stop words and ranking individual matches based on the section gives a much improved user experience.
The old makewhatis and friends are kept under MKMAKEMANDB=no for now.
|
| 1.2.4.6 | 22-May-2014 |
yamt | sync with head.
for a reference, the tree before this commit was tagged as yamt-pagecache-tag8.
this commit was splitted into small chunks to avoid a limitation of cvs. ("Protocol error: too many arguments")
|
| 1.2.4.5 | 23-Jan-2013 |
yamt | sync with head
|
| 1.2.4.4 | 30-Oct-2012 |
yamt | sync with head
|
| 1.2.4.3 | 23-May-2012 |
yamt | sync with head.
|
| 1.2.4.2 | 17-Apr-2012 |
yamt | sync with head
|
| 1.2.4.1 | 07-Feb-2012 |
yamt | file apropos-utils.h was added on branch yamt-pagecache on 2012-04-17 00:09:49 +0000
|
| 1.2.2.1 | 09-May-2012 |
riz | Pull up following revision(s) (requested by wiz in ticket #229): usr.sbin/makemandb/makemandb.c: revision 1.9 usr.sbin/makemandb/DBSCHEMA: revision 1.2 usr.sbin/makemandb/apropos-utils.c: revision 1.5 usr.sbin/makemandb/apropos-utils.h: revision 1.3 PR 46419 by Abhinav Upadhyay using his updated patch: Clean up after removing man page aliases.
|
| 1.3.2.3 | 23-Jun-2013 |
tls | resync from head
|
| 1.3.2.2 | 25-Feb-2013 |
tls | resync with head
|
| 1.3.2.1 | 20-Nov-2012 |
tls | Resync to 2012-11-19 00:00:00 UTC
|
| 1.11.6.1 | 02-May-2017 |
pgoyette | Sync with HEAD - tag prg-localcount2-base1
|
| 1.14.4.1 | 10-Jun-2019 |
christos | Sync with HEAD
|
| 1.23 | 17-May-2022 |
uwe | apropos(1): Use the official spelling for "SQLite".
While here, use .Bx to refer to 3BSD.
|
| 1.22 | 17-May-2022 |
uwe | apropos(1): Tweak the description of -1, ... -9, and -s
-s is not for compatibility only, because section names can be anything. E.g. we have 3lua and 9lua in base. We have rudiments of 3f (for FORTRAN libs). Some packages in pkgsrc also use suffixed 1 and 3 sections.
|
| 1.21 | 17-May-2022 |
uwe | apropos(1): use proper -width for the list of options too
|
| 1.20 | 17-May-2022 |
uwe | apropos(1): use proper -width
|
| 1.19 | 17-May-2022 |
gutteridge | apropos.1: document the PAGER environment variable
|
| 1.18 | 17-Jun-2016 |
abhinav | branches: 1.18.18; Fix grammar/spelling at few places.
|
| 1.17 | 16-Jun-2016 |
abhinav | Add man.conf(5) and man(1) in the SEE ALSO section.
|
| 1.16 | 22-May-2016 |
abhinav | Add options to whatis and apropos to accept custom man.conf.
makemandb(8), man(1) already use -C as an option to take man.conf path, so use the same option for whatis(1) and apropos(1) for consitency. apropos was using -C/-c to disable/enable context of the search matches, change that to -M/-m respectively.
|
| 1.15 | 04-Nov-2014 |
snj | Thankfully, apropos returns all results by default these days. Remove outdated sentence referring to the original (well, old new) limit of 10.
|
| 1.14 | 29-Nov-2013 |
wiz | branches: 1.14.4; Wording improvements from Abhinav Upadhyay <er.abhinav.upadhyay@gmail.com>
|
| 1.13 | 02-Apr-2013 |
wiz | Bump date.
|
| 1.12 | 02-Apr-2013 |
christos | instead of having a format and no format flag, and exposing various formatters, provide a format enum and expose html formatting too.
|
| 1.11 | 29-Mar-2013 |
wiz | Use Ev for environment variables. Improve -n description.
|
| 1.10 | 29-Mar-2013 |
christos | - Fix legacy mode to use like instead of match. This loses ranking. - default to unlimited lines - fix formatting of legacy mode
|
| 1.9 | 29-Mar-2013 |
christos | Don't enumerate all the digit sections Explain limitations in legacy mode
|
| 1.8 | 29-Mar-2013 |
christos | - If the stdout is not a tty, prevent formatting unless forced with -i - Don't ever page unless asked for with -p - Introduce "legacy mode" (-l) 1. searches only name and name_desc, prints name(section) - name_description 2. turns off escape formatting (can be forced on with -i) 3. turns off context printing (can be forced on with -c) - Parse the environment $APROPOS variable as an argument vector.
With these changes one can simply 'export APROPOS=-l' and get the old apropos behavior.
|
| 1.7 | 10-Feb-2013 |
christos | add -r flag to elide tty formatting
|
| 1.6 | 06-Oct-2012 |
wiz | Make mandb path configurable. makemandb (and related tools) use the path from the _mandb variable from man.conf now.
Set _mandb in man.conf to same value as was used before.
From Abhinav Upadhyay <er.abhinav.upadhyay@gmail.com>.
|
| 1.5 | 22-Apr-2012 |
wiz | branches: 1.5.2; Add HISTORY section. From Abhinav Upadhyay in private mail.
|
| 1.4 | 20-Feb-2012 |
joerg | branches: 1.4.2; Add reference to whatis(1). From Abhinav Upadhyay.
|
| 1.3 | 15-Feb-2012 |
joerg | Add support for apropos -s like in the old apropos. Fix capitalisation. Add note about optional quotation. From Abhinav Upadhyay.
|
| 1.2 | 10-Feb-2012 |
njoly | branches: 1.2.2; Fix makemandb section.
|
| 1.1 | 07-Feb-2012 |
joerg | Import the new apropos/whatis.
This code has been developed by Abhinav Upadhyay as part of Google's Summer of Code 2011. It uses libmandoc to parse man pages and builds a Full Text Index in a SQLite database. The combination of indexing the full manual page, filtering out stop words and ranking individual matches based on the section gives a much improved user experience.
The old makewhatis and friends are kept under MKMAKEMANDB=no for now.
|
| 1.2.2.3 | 20-Nov-2012 |
riz | Pull up following revision(s) (requested by tron in ticket #676): usr.sbin/makemandb/apropos.c: revision 1.7 usr.sbin/makemandb/makemandb.8: revision 1.3 usr.sbin/makemandb/makemandb.8: revision 1.4 usr.sbin/makemandb/makemandb.8: revision 1.5 etc/rc.d/makemandb: revision 1.4 etc/weekly: revision 1.29 usr.sbin/makemandb/makemandb.c: revision 1.10 usr.sbin/makemandb/makemandb.c: revision 1.11 usr.sbin/makemandb/makemandb.c: revision 1.12 usr.sbin/makemandb/makemandb.c: revision 1.13 usr.sbin/makemandb/makemandb.c: revision 1.14 usr.sbin/makemandb/apropos.1: revision 1.5 etc/daily: revision 1.87 Add HISTORY section. From Abhinav Upadhyay in private mail. Sync with reality, From Abhinav Upadhyay in private mail. Fix typo in a message. Bug fix for PR 46733: > makemandb always reports the same number for "Total Number of new or > updated pages enountered" and "Total number of (hard or symbolic) > links found". Patch from Abhinav Upadhyay. Sort options in SYNOPSIS. Sync usage with manpage. Add -Q flag: Print only fatal error messages (i.e., when the database is left in an inconsistent state and needs manual intervention). From Abhinav Upadhyay <er.abhinav.upadhyay@gmail.com>. Use new makemandb -Q flag to be really quiet. Fix usage, from Abhinav Upadhyay <er.abhinav.upadhyay@gmail.com>. Use emalloc in one more place, like the rest of the code does. From Abhinav Upadhyay <er.abhinav.upadhyay@gmail.com>.
|
| 1.2.2.2 | 20-Feb-2012 |
sborrill | Pull up the following revisions(s) (requested by joerg in ticket #15): usr.sbin/makemandb/apropos.1: revision 1.4 usr.sbin/makemandb/makemandb.8: revision 1.2 usr.sbin/makemandb/whatis.c: revision 1.3
Add reference to whatis(1). SQLite doesn't use the FTS index for equal ops, so force it to do a FTS search first. Drops run time by a factor of 6 for "whatis man".
|
| 1.2.2.1 | 18-Feb-2012 |
riz | Pull up following revision(s) (requested by joerg in ticket #4): usr.sbin/makemandb/apropos.c: revision 1.5 external/bsd/mdocml/dist/read.c: revision 1.6 external/bsd/mdocml/dist/mandoc.h: revision 1.2 usr.sbin/makemandb/makemandb.c: revision 1.3 usr.sbin/makemandb/makemandb.c: revision 1.4 usr.sbin/makemandb/makemandb.c: revision 1.5 usr.sbin/makemandb/Makefile: revision 1.2 etc/man.conf: revision 1.30 usr.sbin/makemandb/apropos.1: revision 1.3 external/bsd/mdocml/Makefile.inc: revision 1.15 Be a bit more friendly to man pages using the roff .so command by changing the current directory to the parent of the man -p entry, e.g. /usr/share/man for /usr/share/man1. Also handle hyphen replacement if it was used as plain input and no backslash sequence was used at all in the line. Add support for apropos -s like in the old apropos. Fix capitalisation. Add note about optional quotation. From Abhinav Upadhyay. Merge mandoc.h r1.99 and read.c 1.28 from upstream: Add mparse_readmem, which allows application to run the parser directly from memory, e.g. after de-compressing a document. Add support for compressed man pages in all the usual formats. Use mmap when dealing with plain files. Add entries for bzip2 and xz.
|
| 1.4.2.5 | 22-May-2014 |
yamt | sync with head.
for a reference, the tree before this commit was tagged as yamt-pagecache-tag8.
this commit was splitted into small chunks to avoid a limitation of cvs. ("Protocol error: too many arguments")
|
| 1.4.2.4 | 30-Oct-2012 |
yamt | sync with head
|
| 1.4.2.3 | 23-May-2012 |
yamt | sync with head.
|
| 1.4.2.2 | 17-Apr-2012 |
yamt | sync with head
|
| 1.4.2.1 | 20-Feb-2012 |
yamt | file apropos.1 was added on branch yamt-pagecache on 2012-04-17 00:09:49 +0000
|
| 1.5.2.4 | 20-Aug-2014 |
tls | Rebase to HEAD as of a few days ago.
|
| 1.5.2.3 | 23-Jun-2013 |
tls | resync from head
|
| 1.5.2.2 | 25-Feb-2013 |
tls | resync with head
|
| 1.5.2.1 | 20-Nov-2012 |
tls | Resync to 2012-11-19 00:00:00 UTC
|
| 1.14.4.1 | 07-Nov-2014 |
martin | Pull up following revision(s) (requested by snj in ticket #185): usr.sbin/makemandb/apropos.1: revision 1.15 Thankfully, apropos returns all results by default these days. Remove outdated sentence referring to the original (well, old new) limit of 10.
|
| 1.18.18.1 | 03-Jun-2022 |
martin | Pull up following revision(s) (requested by gutteridge in ticket #1461):
usr.sbin/makemandb/apropos.1: revision 1.19 usr.sbin/makemandb/apropos.c: revision 1.25 usr.sbin/makemandb/apropos.c: revision 1.26 usr.sbin/makemandb/apropos.1: revision 1.20 usr.sbin/makemandb/apropos.1: revision 1.21 usr.sbin/makemandb/apropos.1: revision 1.22 usr.sbin/makemandb/apropos.1: revision 1.23 usr.sbin/makemandb/apropos-utils.c: revision 1.46 usr.sbin/makemandb/apropos-utils.c: revision 1.47 usr.sbin/makemandb/apropos-utils.c: revision 1.49
PR/54343: Prevent NULL pointers in callback strings; use "*?*" for now to identify them.
PR bin/54343: We want the callback_args.machine to be NULL if it is not present in the DB.
The previous commit fixed the problem of allowing apropos to not crash and produce output even if the database is missing values for certain mandatory fields, such as name, section etc. Normally we don't expect those values to be missing in the database but in case of parsing errors it can happen.
However, the machine architecture is an optional field since not all man pages are hardware specific so that should be allowed to be set to NULL if not present in the database.
apropos.c: fix pager functionality
Issue reported by Rocky Hotas on NetBSD-Users, patch input from RVP on same, adjustments by me.
apropos.1: document the PAGER environment variable
apropos(1): use proper -width
apropos(1): use proper -width for the list of options too
apropos(1): Tweak the description of -1, ... -9, and -s
-s is not for compatibility only, because section names can be anything. E.g. we have 3lua and 9lua in base. We have rudiments of 3f (for FORTRAN libs). Some packages in pkgsrc also use suffixed 1 and 3 sections.
apropos(1): Use the official spelling for "SQLite". While here, use .Bx to refer to 3BSD.
apropos(1): improve error handling in edge cases Patch from RVP on NetBSD-Users, with an additional comment tweak by me.
Summary from RVP: 1. Ignore SIGPIPE so that we're not killed in the middle of some DB operation by a botched $PAGER: $ env PAGER=3D/non-existent apropos -p ... 2. Return proper exit status in case of write errors: $ apropos ... >/dev/full || echo fail
|
| 1.27 | 03-Aug-2023 |
rin | makemandb: trailing whitespace
|
| 1.26 | 19-May-2022 |
gutteridge | apropos(1): improve error handling in edge cases
Patch from RVP on NetBSD-Users, with an additional comment tweak by me. Summary from RVP:
1. Ignore SIGPIPE so that we're not killed in the middle of some DB operation by a botched $PAGER:
$ env PAGER=/non-existent apropos -p ...
2. Return proper exit status in case of write errors:
$ apropos ... >/dev/full || echo fail
|
| 1.25 | 17-May-2022 |
gutteridge | apropos.c: fix pager functionality
Issue reported by Rocky Hotas on NetBSD-Users, patch input from RVP on same, adjustments by me.
|
| 1.24 | 25-Nov-2017 |
abhinav | branches: 1.24.6; Encapsulate all the arguments required by the query callback function in a struct.
If we want to add or remove arguments from the callback functions, it requires changing the callback interface all over the place. By letting the callback simply expect a single struct argument, it would clean things up a bit.
ok christos
|
| 1.23 | 02-Aug-2017 |
jmcneill | Make the 'no results found' message sound less harsh.
Changes "try using better keywords" to "try using different keywords".
|
| 1.22 | 01-May-2017 |
abhinav | branches: 1.22.2; Simplify handling of the section arguments in apropos(1).
Earlier, a white space separated string was generated containing all the section numbers passed through command line arguments. Later on that would have to be tokenized and processed. Instead of that, use a NULL terminated array of strings.
Thanks to christos@ for reviewing and suggesting further improvements.
|
| 1.21 | 22-May-2016 |
abhinav | branches: 1.21.6; Add options to whatis and apropos to accept custom man.conf.
makemandb(8), man(1) already use -C as an option to take man.conf path, so use the same option for whatis(1) and apropos(1) for consitency. apropos was using -C/-c to disable/enable context of the search matches, change that to -M/-m respectively.
|
| 1.20 | 23-Apr-2016 |
christos | add a newline
|
| 1.19 | 13-Apr-2016 |
christos | PR/51062: Abhinav Upadhyay: Allow non numeric sections to be indexed and searched by apropos(1). Fold long lines.
|
| 1.18 | 31-Mar-2016 |
christos | PR/51007: Abhinav Upadhyay: apropos.c: Remove unused includes
|
| 1.17 | 20-Dec-2015 |
christos | if we only asked for stopwords, use the original query.
|
| 1.16 | 02-Apr-2013 |
christos | instead of having a format and no format flag, and exposing various formatters, provide a format enum and expose html formatting too.
|
| 1.15 | 29-Mar-2013 |
wiz | Sync usage with man page.
|
| 1.14 | 29-Mar-2013 |
christos | - Fix legacy mode to use like instead of match. This loses ranking. - default to unlimited lines - fix formatting of legacy mode
|
| 1.13 | 29-Mar-2013 |
christos | - If the stdout is not a tty, prevent formatting unless forced with -i - Don't ever page unless asked for with -p - Introduce "legacy mode" (-l) 1. searches only name and name_desc, prints name(section) - name_description 2. turns off escape formatting (can be forced on with -i) 3. turns off context printing (can be forced on with -c) - Parse the environment $APROPOS variable as an argument vector.
With these changes one can simply 'export APROPOS=-l' and get the old apropos behavior.
|
| 1.12 | 10-Feb-2013 |
christos | add -r flag to elide tty formatting
|
| 1.11 | 10-Feb-2013 |
christos | remove trailing whitespace
|
| 1.10 | 14-Jan-2013 |
christos | - move the terminal handling in apropos-utils.c since htmp and pager are also handled there. - underline the name, section, and description so that it is prettier. - change to bold terminal the terminal highlighting to match with less
|
| 1.9 | 14-Jan-2013 |
christos | Fix the #ifdef NOTYET, namely use the terminal standout mode to highlight the search terms.
|
| 1.8 | 06-Oct-2012 |
wiz | Make mandb path configurable. makemandb (and related tools) use the path from the _mandb variable from man.conf now.
Set _mandb in man.conf to same value as was used before.
From Abhinav Upadhyay <er.abhinav.upadhyay@gmail.com>.
|
| 1.7 | 07-Sep-2012 |
wiz | branches: 1.7.2; Fix usage, from Abhinav Upadhyay <er.abhinav.upadhyay@gmail.com>.
|
| 1.6 | 15-Apr-2012 |
wiz | branches: 1.6.2; Handle pages with slashes in their names better. From Abhinav Upadhyay in private mail.
|
| 1.5 | 15-Feb-2012 |
joerg | Add support for apropos -s like in the old apropos. Fix capitalisation. Add note about optional quotation. From Abhinav Upadhyay.
|
| 1.4 | 07-Feb-2012 |
joerg | branches: 1.4.2; Kill bad magic invocation.
|
| 1.3 | 07-Feb-2012 |
joerg | getopt returns int, not char.
|
| 1.2 | 07-Feb-2012 |
joerg | Fix C&P error with $NetBSD$
|
| 1.1 | 07-Feb-2012 |
joerg | Import the new apropos/whatis.
This code has been developed by Abhinav Upadhyay as part of Google's Summer of Code 2011. It uses libmandoc to parse man pages and builds a Full Text Index in a SQLite database. The combination of indexing the full manual page, filtering out stop words and ranking individual matches based on the section gives a much improved user experience.
The old makewhatis and friends are kept under MKMAKEMANDB=no for now.
|
| 1.4.2.3 | 20-Nov-2012 |
riz | Pull up following revision(s) (requested by tron in ticket #676): usr.sbin/makemandb/apropos.c: revision 1.7 usr.sbin/makemandb/makemandb.8: revision 1.3 usr.sbin/makemandb/makemandb.8: revision 1.4 usr.sbin/makemandb/makemandb.8: revision 1.5 etc/rc.d/makemandb: revision 1.4 etc/weekly: revision 1.29 usr.sbin/makemandb/makemandb.c: revision 1.10 usr.sbin/makemandb/makemandb.c: revision 1.11 usr.sbin/makemandb/makemandb.c: revision 1.12 usr.sbin/makemandb/makemandb.c: revision 1.13 usr.sbin/makemandb/makemandb.c: revision 1.14 usr.sbin/makemandb/apropos.1: revision 1.5 etc/daily: revision 1.87 Add HISTORY section. From Abhinav Upadhyay in private mail. Sync with reality, From Abhinav Upadhyay in private mail. Fix typo in a message. Bug fix for PR 46733: > makemandb always reports the same number for "Total Number of new or > updated pages enountered" and "Total number of (hard or symbolic) > links found". Patch from Abhinav Upadhyay. Sort options in SYNOPSIS. Sync usage with manpage. Add -Q flag: Print only fatal error messages (i.e., when the database is left in an inconsistent state and needs manual intervention). From Abhinav Upadhyay <er.abhinav.upadhyay@gmail.com>. Use new makemandb -Q flag to be really quiet. Fix usage, from Abhinav Upadhyay <er.abhinav.upadhyay@gmail.com>. Use emalloc in one more place, like the rest of the code does. From Abhinav Upadhyay <er.abhinav.upadhyay@gmail.com>.
|
| 1.4.2.2 | 19-Apr-2012 |
riz | Pull up following revision(s) (requested by wiz in ticket #186): usr.sbin/makemandb/apropos.c: revision 1.6 usr.sbin/makemandb/apropos-utils.c: revision 1.3 usr.sbin/makemandb/apropos-utils.c: revision 1.4 Add the result from sqlite3_errmsg() to some error messages. Now we can get "apropos: Unable to query schema version: database is locked" instead of just "apropos: Unable to query schema version". Handle pages with slashes in their names better. From Abhinav Upadhyay in private mail.
|
| 1.4.2.1 | 18-Feb-2012 |
riz | Pull up following revision(s) (requested by joerg in ticket #4): usr.sbin/makemandb/apropos.c: revision 1.5 external/bsd/mdocml/dist/read.c: revision 1.6 external/bsd/mdocml/dist/mandoc.h: revision 1.2 usr.sbin/makemandb/makemandb.c: revision 1.3 usr.sbin/makemandb/makemandb.c: revision 1.4 usr.sbin/makemandb/makemandb.c: revision 1.5 usr.sbin/makemandb/Makefile: revision 1.2 etc/man.conf: revision 1.30 usr.sbin/makemandb/apropos.1: revision 1.3 external/bsd/mdocml/Makefile.inc: revision 1.15 Be a bit more friendly to man pages using the roff .so command by changing the current directory to the parent of the man -p entry, e.g. /usr/share/man for /usr/share/man1. Also handle hyphen replacement if it was used as plain input and no backslash sequence was used at all in the line. Add support for apropos -s like in the old apropos. Fix capitalisation. Add note about optional quotation. From Abhinav Upadhyay. Merge mandoc.h r1.99 and read.c 1.28 from upstream: Add mparse_readmem, which allows application to run the parser directly from memory, e.g. after de-compressing a document. Add support for compressed man pages in all the usual formats. Use mmap when dealing with plain files. Add entries for bzip2 and xz.
|
| 1.6.2.5 | 22-May-2014 |
yamt | sync with head.
for a reference, the tree before this commit was tagged as yamt-pagecache-tag8.
this commit was splitted into small chunks to avoid a limitation of cvs. ("Protocol error: too many arguments")
|
| 1.6.2.4 | 23-Jan-2013 |
yamt | sync with head
|
| 1.6.2.3 | 30-Oct-2012 |
yamt | sync with head
|
| 1.6.2.2 | 17-Apr-2012 |
yamt | sync with head
|
| 1.6.2.1 | 15-Apr-2012 |
yamt | file apropos.c was added on branch yamt-pagecache on 2012-04-17 00:09:49 +0000
|
| 1.7.2.3 | 23-Jun-2013 |
tls | resync from head
|
| 1.7.2.2 | 25-Feb-2013 |
tls | resync with head
|
| 1.7.2.1 | 20-Nov-2012 |
tls | Resync to 2012-11-19 00:00:00 UTC
|
| 1.21.6.1 | 02-May-2017 |
pgoyette | Sync with HEAD - tag prg-localcount2-base1
|
| 1.22.2.1 | 05-Aug-2017 |
snj | Pull up following revision(s) (requested by jmcneill in ticket #176): usr.sbin/makemandb/apropos.c: revision 1.23 Make the 'no results found' message sound less harsh. Changes "try using better keywords" to "try using different keywords".
|
| 1.24.6.1 | 03-Jun-2022 |
martin | Pull up following revision(s) (requested by gutteridge in ticket #1461):
usr.sbin/makemandb/apropos.1: revision 1.19 usr.sbin/makemandb/apropos.c: revision 1.25 usr.sbin/makemandb/apropos.c: revision 1.26 usr.sbin/makemandb/apropos.1: revision 1.20 usr.sbin/makemandb/apropos.1: revision 1.21 usr.sbin/makemandb/apropos.1: revision 1.22 usr.sbin/makemandb/apropos.1: revision 1.23 usr.sbin/makemandb/apropos-utils.c: revision 1.46 usr.sbin/makemandb/apropos-utils.c: revision 1.47 usr.sbin/makemandb/apropos-utils.c: revision 1.49
PR/54343: Prevent NULL pointers in callback strings; use "*?*" for now to identify them.
PR bin/54343: We want the callback_args.machine to be NULL if it is not present in the DB.
The previous commit fixed the problem of allowing apropos to not crash and produce output even if the database is missing values for certain mandatory fields, such as name, section etc. Normally we don't expect those values to be missing in the database but in case of parsing errors it can happen.
However, the machine architecture is an optional field since not all man pages are hardware specific so that should be allowed to be set to NULL if not present in the database.
apropos.c: fix pager functionality
Issue reported by Rocky Hotas on NetBSD-Users, patch input from RVP on same, adjustments by me.
apropos.1: document the PAGER environment variable
apropos(1): use proper -width
apropos(1): use proper -width for the list of options too
apropos(1): Tweak the description of -1, ... -9, and -s
-s is not for compatibility only, because section names can be anything. E.g. we have 3lua and 9lua in base. We have rudiments of 3f (for FORTRAN libs). Some packages in pkgsrc also use suffixed 1 and 3 sections.
apropos(1): Use the official spelling for "SQLite". While here, use .Bx to refer to 3BSD.
apropos(1): improve error handling in edge cases Patch from RVP on NetBSD-Users, with an additional comment tweak by me.
Summary from RVP: 1. Ignore SIGPIPE so that we're not killed in the middle of some DB operation by a botched $PAGER: $ env PAGER=3D/non-existent apropos -p ... 2. Return proper exit status in case of write errors: $ apropos ... >/dev/full || echo fail
|
| 1.3 | 11-Sep-2022 |
gutteridge | makemandb/*: fix spelling of database and consistency of SQLite
|
| 1.2 | 24-May-2016 |
abhinav | Bring man pages in sync with reality.
Remove man pages run_query_html.3 and run_query_pager.3 as the corresponding functions have been removed from apropos-utils.c Ok by wiz@
|
| 1.1 | 07-Feb-2012 |
joerg | branches: 1.1.4; Import the new apropos/whatis.
This code has been developed by Abhinav Upadhyay as part of Google's Summer of Code 2011. It uses libmandoc to parse man pages and builds a Full Text Index in a SQLite database. The combination of indexing the full manual page, filtering out stop words and ranking individual matches based on the section gives a much improved user experience.
The old makewhatis and friends are kept under MKMAKEMANDB=no for now.
|
| 1.1.4.2 | 17-Apr-2012 |
yamt | sync with head
|
| 1.1.4.1 | 07-Feb-2012 |
yamt | file close_db.3 was added on branch yamt-pagecache on 2012-04-17 00:09:49 +0000
|
| 1.6 | 07-Aug-2023 |
tnn | makemakedb: don't return uninitialized token length if stemming fails
|
| 1.5 | 03-Aug-2023 |
rin | makemandb: trailing whitespace
|
| 1.4 | 05-Dec-2021 |
wiz | preceds -> precedes
|
| 1.3 | 05-Dec-2021 |
msaitoh | s/preceed/preced/ in comment.
|
| 1.2 | 31-Oct-2017 |
abhinav | Casting variable of type int * to size_t *, may cause alignment issues on some platforms (e.g. Sparc64) So, Use a temporary variable to avoid the cast.
Thanks to Martin@ for noticing the issue and also suggesting the issue. Fixes PR bin/52678
|
| 1.1 | 18-Jun-2017 |
abhinav | Add a custom tokenizer which does not stem certain keywords.
Which keywords should not be stemmed is specified in the nostem.txt file. (Right now I have taken all the man page names, split them if they had underscores, removed common English words and converted everything to lowercase.)
The tokenizer itself is based on the Porter stemming tokenizer shipped with Sqlite. The code in custom_apropos_tokenizer.c is copy of that code with some modifications to prevent stemming keywords specified in nostem.txt.
Additionally, it now uses underscore `_' also as a token delimiter. Therefore, now it's possible to do query for `lwp' and all `_lwp_*' man page names will be matched. Or the query can be `unconst' and `__UNCONST' will be matched. This was not possible earlier, because underscore was not a delimiter and therefore the index would have __UNCONST as a key rather than UNCONST.
The tokenizer needs fts3_tokenizer.h file, which is not shipped with the amalgamation build of Sqlite, therefore it needs to be added here (unless we decide there is a better place for it).
To enforce using the new tokenizer, a schema version bump is needed
Since the tokenization is done both at the indexing time (via makemandb) and also while query time (via apropos or whatis), it will be needed to bump the schema version everytime nostem.txt is modified. Otherwise the index will consist of old tokens and desired changes will not be seen with apropos.
This should also fix the issue reported in PR bin/46255. Similar suggestion was also made on tech-userlevel@ recently: <http://mail-index.netbsd.org/tech-userlevel/2017/06/08/msg010620.html>
Thanks to christos@ for multiple rounds of reviews of the tokenizer code.
|
| 1.1 | 18-Jun-2017 |
abhinav | Add a custom tokenizer which does not stem certain keywords.
Which keywords should not be stemmed is specified in the nostem.txt file. (Right now I have taken all the man page names, split them if they had underscores, removed common English words and converted everything to lowercase.)
The tokenizer itself is based on the Porter stemming tokenizer shipped with Sqlite. The code in custom_apropos_tokenizer.c is copy of that code with some modifications to prevent stemming keywords specified in nostem.txt.
Additionally, it now uses underscore `_' also as a token delimiter. Therefore, now it's possible to do query for `lwp' and all `_lwp_*' man page names will be matched. Or the query can be `unconst' and `__UNCONST' will be matched. This was not possible earlier, because underscore was not a delimiter and therefore the index would have __UNCONST as a key rather than UNCONST.
The tokenizer needs fts3_tokenizer.h file, which is not shipped with the amalgamation build of Sqlite, therefore it needs to be added here (unless we decide there is a better place for it).
To enforce using the new tokenizer, a schema version bump is needed
Since the tokenization is done both at the indexing time (via makemandb) and also while query time (via apropos or whatis), it will be needed to bump the schema version everytime nostem.txt is modified. Otherwise the index will consist of old tokens and desired changes will not be seen with apropos.
This should also fix the issue reported in PR bin/46255. Similar suggestion was also made on tech-userlevel@ recently: <http://mail-index.netbsd.org/tech-userlevel/2017/06/08/msg010620.html>
Thanks to christos@ for multiple rounds of reviews of the tokenizer code.
|
| 1.2 | 03-Aug-2023 |
rin | makemandb: trailing whitespace
|
| 1.1 | 18-Jun-2017 |
abhinav | Add a custom tokenizer which does not stem certain keywords.
Which keywords should not be stemmed is specified in the nostem.txt file. (Right now I have taken all the man page names, split them if they had underscores, removed common English words and converted everything to lowercase.)
The tokenizer itself is based on the Porter stemming tokenizer shipped with Sqlite. The code in custom_apropos_tokenizer.c is copy of that code with some modifications to prevent stemming keywords specified in nostem.txt.
Additionally, it now uses underscore `_' also as a token delimiter. Therefore, now it's possible to do query for `lwp' and all `_lwp_*' man page names will be matched. Or the query can be `unconst' and `__UNCONST' will be matched. This was not possible earlier, because underscore was not a delimiter and therefore the index would have __UNCONST as a key rather than UNCONST.
The tokenizer needs fts3_tokenizer.h file, which is not shipped with the amalgamation build of Sqlite, therefore it needs to be added here (unless we decide there is a better place for it).
To enforce using the new tokenizer, a schema version bump is needed
Since the tokenization is done both at the indexing time (via makemandb) and also while query time (via apropos or whatis), it will be needed to bump the schema version everytime nostem.txt is modified. Otherwise the index will consist of old tokens and desired changes will not be seen with apropos.
This should also fix the issue reported in PR bin/46255. Similar suggestion was also made on tech-userlevel@ recently: <http://mail-index.netbsd.org/tech-userlevel/2017/06/08/msg010620.html>
Thanks to christos@ for multiple rounds of reviews of the tokenizer code.
|
| 1.4 | 11-Sep-2022 |
gutteridge | makemandb/*: fix spelling of database and consistency of SQLite
|
| 1.3 | 24-May-2016 |
abhinav | Bring man pages in sync with reality.
Remove man pages run_query_html.3 and run_query_pager.3 as the corresponding functions have been removed from apropos-utils.c Ok by wiz@
|
| 1.2 | 06-Oct-2012 |
wiz | Make mandb path configurable. makemandb (and related tools) use the path from the _mandb variable from man.conf now.
Set _mandb in man.conf to same value as was used before.
From Abhinav Upadhyay <er.abhinav.upadhyay@gmail.com>.
|
| 1.1 | 07-Feb-2012 |
joerg | branches: 1.1.4; 1.1.8; Import the new apropos/whatis.
This code has been developed by Abhinav Upadhyay as part of Google's Summer of Code 2011. It uses libmandoc to parse man pages and builds a Full Text Index in a SQLite database. The combination of indexing the full manual page, filtering out stop words and ranking individual matches based on the section gives a much improved user experience.
The old makewhatis and friends are kept under MKMAKEMANDB=no for now.
|
| 1.1.8.1 | 20-Nov-2012 |
tls | Resync to 2012-11-19 00:00:00 UTC
|
| 1.1.4.3 | 30-Oct-2012 |
yamt | sync with head
|
| 1.1.4.2 | 17-Apr-2012 |
yamt | sync with head
|
| 1.1.4.1 | 07-Feb-2012 |
yamt | file init_db.3 was added on branch yamt-pagecache on 2012-04-17 00:09:49 +0000
|
| 1.11 | 11-Sep-2022 |
gutteridge | makemandb/*: fix spelling of database and consistency of SQLite
|
| 1.10 | 18-Jun-2016 |
abhinav | Improve wording, and add reference to mandoc(3) Ok from wiz@
|
| 1.9 | 22-May-2016 |
abhinav | Remove a trailing white space after .Nm (mandoc -Tlint was complaining about it)
|
| 1.8 | 03-Mar-2015 |
christos | mention that this replaces makewhatis(8)
|
| 1.7 | 29-Nov-2013 |
wiz | branches: 1.7.4; Wording improvements from Abhinav Upadhyay <er.abhinav.upadhyay@gmail.com>
|
| 1.6 | 06-Oct-2012 |
wiz | Make mandb path configurable. makemandb (and related tools) use the path from the _mandb variable from man.conf now.
Set _mandb in man.conf to same value as was used before.
From Abhinav Upadhyay <er.abhinav.upadhyay@gmail.com>.
|
| 1.5 | 29-Aug-2012 |
wiz | branches: 1.5.2; Add -Q flag: Print only fatal error messages (i.e., when the database is left in an inconsistent state and needs manual intervention).
From Abhinav Upadhyay <er.abhinav.upadhyay@gmail.com>.
|
| 1.4 | 25-Aug-2012 |
wiz | Sort options in SYNOPSIS.
|
| 1.3 | 22-Apr-2012 |
wiz | Sync with reality, From Abhinav Upadhyay in private mail.
|
| 1.2 | 20-Feb-2012 |
joerg | branches: 1.2.2; Add reference to whatis(1). From Abhinav Upadhyay.
|
| 1.1 | 07-Feb-2012 |
joerg | branches: 1.1.2; Import the new apropos/whatis.
This code has been developed by Abhinav Upadhyay as part of Google's Summer of Code 2011. It uses libmandoc to parse man pages and builds a Full Text Index in a SQLite database. The combination of indexing the full manual page, filtering out stop words and ranking individual matches based on the section gives a much improved user experience.
The old makewhatis and friends are kept under MKMAKEMANDB=no for now.
|
| 1.1.2.2 | 20-Nov-2012 |
riz | Pull up following revision(s) (requested by tron in ticket #676): usr.sbin/makemandb/apropos.c: revision 1.7 usr.sbin/makemandb/makemandb.8: revision 1.3 usr.sbin/makemandb/makemandb.8: revision 1.4 usr.sbin/makemandb/makemandb.8: revision 1.5 etc/rc.d/makemandb: revision 1.4 etc/weekly: revision 1.29 usr.sbin/makemandb/makemandb.c: revision 1.10 usr.sbin/makemandb/makemandb.c: revision 1.11 usr.sbin/makemandb/makemandb.c: revision 1.12 usr.sbin/makemandb/makemandb.c: revision 1.13 usr.sbin/makemandb/makemandb.c: revision 1.14 usr.sbin/makemandb/apropos.1: revision 1.5 etc/daily: revision 1.87 Add HISTORY section. From Abhinav Upadhyay in private mail. Sync with reality, From Abhinav Upadhyay in private mail. Fix typo in a message. Bug fix for PR 46733: > makemandb always reports the same number for "Total Number of new or > updated pages enountered" and "Total number of (hard or symbolic) > links found". Patch from Abhinav Upadhyay. Sort options in SYNOPSIS. Sync usage with manpage. Add -Q flag: Print only fatal error messages (i.e., when the database is left in an inconsistent state and needs manual intervention). From Abhinav Upadhyay <er.abhinav.upadhyay@gmail.com>. Use new makemandb -Q flag to be really quiet. Fix usage, from Abhinav Upadhyay <er.abhinav.upadhyay@gmail.com>. Use emalloc in one more place, like the rest of the code does. From Abhinav Upadhyay <er.abhinav.upadhyay@gmail.com>.
|
| 1.1.2.1 | 20-Feb-2012 |
sborrill | Pull up the following revisions(s) (requested by joerg in ticket #15): usr.sbin/makemandb/apropos.1: revision 1.4 usr.sbin/makemandb/makemandb.8: revision 1.2 usr.sbin/makemandb/whatis.c: revision 1.3
Add reference to whatis(1). SQLite doesn't use the FTS index for equal ops, so force it to do a FTS search first. Drops run time by a factor of 6 for "whatis man".
|
| 1.2.2.5 | 22-May-2014 |
yamt | sync with head.
for a reference, the tree before this commit was tagged as yamt-pagecache-tag8.
this commit was splitted into small chunks to avoid a limitation of cvs. ("Protocol error: too many arguments")
|
| 1.2.2.4 | 30-Oct-2012 |
yamt | sync with head
|
| 1.2.2.3 | 23-May-2012 |
yamt | sync with head.
|
| 1.2.2.2 | 17-Apr-2012 |
yamt | sync with head
|
| 1.2.2.1 | 20-Feb-2012 |
yamt | file makemandb.8 was added on branch yamt-pagecache on 2012-04-17 00:09:49 +0000
|
| 1.5.2.2 | 20-Aug-2014 |
tls | Rebase to HEAD as of a few days ago.
|
| 1.5.2.1 | 20-Nov-2012 |
tls | Resync to 2012-11-19 00:00:00 UTC
|
| 1.7.4.1 | 14-Apr-2015 |
snj | Pull up following revision(s) (requested by christos in ticket #676): usr.bin/man/man.conf.5: revision 1.25 usr.sbin/makemandb/makemandb.8: revision 1.8 mention that this replaces makewhatis(8) -- there is no more makewhatis; it has been replaced by makemandb
|
| 1.67 | 01-Jan-2023 |
gutteridge | makemandb.c: spell "metadata" consistently
|
| 1.66 | 30-Oct-2022 |
gutteridge | makemandb.c: fix grammar in a comment
|
| 1.65 | 26-Oct-2022 |
andvar | fix various typos in comments and makefs README file.
|
| 1.64 | 11-Sep-2022 |
gutteridge | makemandb/*: fix spelling of database and consistency of SQLite
|
| 1.63 | 06-Jun-2022 |
skrll | Don't index outside the mdocs array of function pointers. Analysis and suggested fixes from Tom Lane. I played it safe and went with (my variation of) the minimal fix.
port-hppa/56118: sporadic app crashes in HPPA -current
|
| 1.62 | 06-Apr-2022 |
gutteridge | makemandb.c: fail sooner if man page dirs can't be found
There's no point initializing database state if we're then going to fail to locate any man page sources. Make all the initial state checks contiguous for simplicity and readability. Also, free the variable "command" on the error path, and correct the error message.
|
| 1.61 | 05-Dec-2021 |
msaitoh | s/trival/trivial/ in comment.
|
| 1.60 | 18-May-2019 |
abhinav | branches: 1.60.2; PR misc/54213: Fix performance of whatis(1) when no matches are found
In revision 1.6 of whatis.c the query was modified to return matches for names found in MLINKS of the man pages as well. However it was slow. The reason probably being that it required a join. But more importantly the where condition on an FTS virtual table column is very slow. To avoid the join and the expensive where condition on the virtual table, add the name_desc column to the mandb_links table as well. This improves the performance of whatis(1) to the original level at the expense of slight data duplication.
Bump the schema to force database rebuild to take account for the new column addition
|
| 1.59 | 11-Mar-2019 |
christos | remove unneeded header.
|
| 1.58 | 11-Mar-2019 |
christos | adjust to the new mandoc api
|
| 1.57 | 24-Aug-2018 |
abhinav | Adjust makemandb for the latest mandoc
ok christos@
|
| 1.56 | 16-Aug-2018 |
kre | In the latest mandoc (mdocml) the signature (prototype) of mparse_alloc() altered - update the call here to compensate.
This fixes the build (of makemandb), but I am not sure that the changed version is what is desired - someone who knows something about all of this should validate ... I just copied the invocation from mandoc's demandoc.c (which seems likely to be at least a similar kind of usage).
|
| 1.55 | 10-May-2017 |
abhinav | branches: 1.55.8; 1.55.10; Get rid of unnecessary variable.
|
| 1.54 | 02-May-2017 |
abhinav | We do need to copy the return value from dirname(3) since there it is a static buffer and can be overwritten in between. I overzealously removed this in one of my previous commits.
|
| 1.53 | 01-May-2017 |
abhinav | Avoid dereferencing pointer at multiple places, instead use a local variable.
|
| 1.52 | 01-May-2017 |
abhinav | Remove the table name parameter from the check_md5 function.
There is only one table storing the md5 checksums, so we can hardcode the table name instead of passing it as a function argument.
|
| 1.51 | 01-May-2017 |
abhinav | Avoid copying strings where it is not needed.
|
| 1.50 | 30-Apr-2017 |
abhinav | Avoid a call to strncmp when comparing only the first character of the string.
|
| 1.49 | 29-Apr-2017 |
abhinav | Bring the comment in sync with code (after changes brought by the last commit).
|
| 1.48 | 29-Apr-2017 |
abhinav | Don't parse Nm macro when it occurs anywhere outside the NAME section.
mandoc(3) already generates the text node representing the value for the .Nm macro. Doing our own parsing for .Nm on top of that leads to large duplication of text in the database. This gets specially worse for man pages with large NAME sections, such as queue(3).
|
| 1.47 | 20-Apr-2017 |
joerg | branches: 1.47.2; Use libarchive 3.x interface and not obsolete 2.x versions.
|
| 1.46 | 19-Dec-2016 |
abhinav | branches: 1.46.2; Escape hyphen when parsing .Nd
|
| 1.45 | 17-Dec-2016 |
abhinav | Don't ignore symlinks. There can be symlinks which are pointing to man pages not installed in one of the _default locations mentioned in man.conf or MANPATH. For example there are man pages in /usr/pkg/man which are symlinked to pages in /usr/pkg/lib/perl5/man. If we ignore symlinks, we would not be able to index such pages installed outside the default set of directories.
(Also, the symlink test was incorecct, so we never noticed this issue)
Ok christos@, wiz@
|
| 1.44 | 03-Oct-2016 |
abhinav | We don't need to parse the sections we don't index, so stop early. Saves few instructions.
|
| 1.43 | 03-Oct-2016 |
abhinav | With the latest release of mandoc, makemandb(8) started to parse some sections multiple times. This started to happen because, pmdoc_Sh(), the handler function responsible for parsing the Sh macros, used to recursively go through all the child nodes and then the next nodes starting from top level Sh block node. Now, once it has processed all the child nodes of the top level block node, it moves to the next node, which is the top level block node of the next section and in this way one call to pmdoc_Sh() was causing a complete pass through the man page. Since, mandoc(3) calls pmdoc_Sh() for each .Sh macro in the man page, it would result in parsing some of the sections multiple times. This never happened with the previous versions of mandoc, so we never noticed.
I've fixed this by starting the parse sequence of the Sh macro from its body, which gurantees that we will stop once that section ends.
ok christos@
|
| 1.42 | 17-Jul-2016 |
abhinav | Use deroff() from mandoc(3) to directly parse the Nd macro rather than parsing it by hand.
With the latest mandoc(3), the .Nd macro was getting parsed twice. This fixes that problem and cleans up the code as well.
ok christos@
|
| 1.41 | 17-Jul-2016 |
abhinav | Fix extraction of section number and machine architecture from man pages. With the latest API, mdoc_validate()/man_validate() needs to be called before reading the roff_man.meta field, otherwise it is NULL.
Also, if a man page doesn't specify machine architecture, don't default to '?' , let it be stored as null in the db. Otherwise, the output of apropos(1) shows the names of the results as \?/<title>
|
| 1.40 | 15-Jul-2016 |
christos | Sync with API changes.
|
| 1.39 | 06-Jul-2016 |
abhinav | branches: 1.39.2; Avoid possible buffer overflow while parsing NAME section of man(7) pages. Also, simplify copyging of strings, use estrdupn instead of emalloc + memcpy.
Patch from christos@, XXX comment by me
|
| 1.38 | 05-Jul-2016 |
abhinav | Reuse variable from previous line.
|
| 1.37 | 13-Apr-2016 |
christos | PR/51062: Abhinav Upadhyay: Allow non numeric sections to be indexed and searched by apropos(1). Fold long lines.
|
| 1.36 | 13-Apr-2016 |
christos | PR/51040: Abhinav Upadhyay: Fix memory leak
|
| 1.35 | 13-Apr-2016 |
christos | PR/51039: Abhinav Upadhyay: Check for return value of chdir(2)
|
| 1.34 | 13-Apr-2016 |
christos | PR/51034: Abhinav Upadhyay: Close database connection when failed to commit
|
| 1.33 | 31-Mar-2016 |
christos | PR/51034: Abhinav Upadhyay: makemandb(8): Close database connection when failed to commit
|
| 1.32 | 24-Mar-2016 |
christos | PR/51006: Abhinav Upadhyay: makemandb(8) should parse escape sequences in the NAME section
|
| 1.31 | 28-Jan-2016 |
christos | Don't crash if we have a missing section.
|
| 1.30 | 18-Dec-2015 |
christos | Adjust to the new mdocml
|
| 1.29 | 07-Apr-2015 |
plunky | largely apply patch from PR bin/47392 by Abhinav Upadhyay
change some comments to reflect reality, a variable name to enhance readability, and adds an assert for safety.
|
| 1.28 | 12-Mar-2015 |
joerg | MDOC_MAX is a valid token if the type is text. Adjust.
|
| 1.27 | 04-Mar-2015 |
christos | - handle section numbers that are not single digits - don't allocate and free needlessly
|
| 1.26 | 02-Mar-2015 |
joerg | Explicitly deal with end of lists. PR 49708.
|
| 1.25 | 18-Oct-2014 |
snj | src is too big these days to tolerate superfluous apostrophes. It's "its", people!
|
| 1.24 | 03-Jun-2014 |
wiz | branches: 1.24.2; Fix a bug that caused an error about a UNIQUE constraint violation. Patch from Abhinav Upadhyay.
|
| 1.23 | 24-May-2014 |
wiz | Replace non-breaking space with hyphen, and call hyphen replacement from one more place. Improves 'man -k midi' output.
From Abhinav Upadhyay.
|
| 1.22 | 10-Feb-2014 |
chs | branches: 1.22.2; in update_db(), extract the full list of files to update from the db before actually updating anything, since changing the db while the query that extracts the list of files is still in progress results in the extraction query failing before it finds everything.
|
| 1.21 | 05-Jan-2014 |
joerg | Sync with interface change in mdocml 1.12.3.
|
| 1.20 | 13-Nov-2013 |
wiz | Skip files of size 0 from indexing.
From Abhinav Upadhyay <er.abhinav.upadhyay@gmail.com>.
|
| 1.19 | 15-May-2013 |
christos | avoid stdio assertion, failing later
|
| 1.18 | 10-Feb-2013 |
christos | remove trailing whitespace
|
| 1.17 | 14-Jan-2013 |
christos | Since mdocml decided to name headers that conflict with system ones (term.h) move the header inclusion one up.
|
| 1.16 | 08-Nov-2012 |
christos | If you cannot parse .SH NAME, like in the case of the ksh93 man page where the .SH is followed by a conditional:
.SH NAME .if \nZ=0 \{\ text text text .\}
at least don't core-dump.
|
| 1.15 | 06-Oct-2012 |
wiz | Make mandb path configurable. makemandb (and related tools) use the path from the _mandb variable from man.conf now.
Set _mandb in man.conf to same value as was used before.
From Abhinav Upadhyay <er.abhinav.upadhyay@gmail.com>.
|
| 1.14 | 07-Sep-2012 |
wiz | branches: 1.14.2; Use emalloc in one more place, like the rest of the code does. From Abhinav Upadhyay <er.abhinav.upadhyay@gmail.com>.
|
| 1.13 | 29-Aug-2012 |
wiz | Add -Q flag: Print only fatal error messages (i.e., when the database is left in an inconsistent state and needs manual intervention).
From Abhinav Upadhyay <er.abhinav.upadhyay@gmail.com>.
|
| 1.12 | 25-Aug-2012 |
wiz | Sync usage with manpage.
|
| 1.11 | 11-Aug-2012 |
wiz | Bug fix for PR 46733: > makemandb always reports the same number for "Total Number of new or > updated pages enountered" and "Total number of (hard or symbolic) > links found".
Patch from Abhinav Upadhyay.
|
| 1.10 | 08-Jul-2012 |
uwe | Fix typo in a message.
|
| 1.9 | 07-May-2012 |
wiz | PR 46419 by Abhinav Upadhyay using his updated patch: Clean up after removing man page aliases.
|
| 1.8 | 04-May-2012 |
wiz | The new apropos(1) incorrectly displays hyphens in the first line of the search results for a few man pages (for man(7) based man pages).
Use patch from Abhinav Upadhyay in PR 46408 to fix this.
|
| 1.7 | 02-Mar-2012 |
joerg | branches: 1.7.2; Fix inverted condition when handling stale entries. From Abhinav Upadhyay.
|
| 1.6 | 27-Feb-2012 |
joerg | Expand workaround for .so usage to do the chdir call just before starting parsing, not during the tree iteration. This gives it a chance to work.
|
| 1.5 | 16-Feb-2012 |
joerg | Add support for compressed man pages in all the usual formats.
|
| 1.4 | 15-Feb-2012 |
joerg | Also handle hyphen replacement if it was used as plain input and no backslash sequence was used at all in the line.
|
| 1.3 | 15-Feb-2012 |
joerg | Be a bit more friendly to man pages using the roff .so command by changing the current directory to the parent of the man -p entry, e.g. /usr/share/man for /usr/share/man1.
|
| 1.2 | 07-Feb-2012 |
joerg | branches: 1.2.2; Fix C&P error with $NetBSD$
|
| 1.1 | 07-Feb-2012 |
joerg | Import the new apropos/whatis.
This code has been developed by Abhinav Upadhyay as part of Google's Summer of Code 2011. It uses libmandoc to parse man pages and builds a Full Text Index in a SQLite database. The combination of indexing the full manual page, filtering out stop words and ranking individual matches based on the section gives a much improved user experience.
The old makewhatis and friends are kept under MKMAKEMANDB=no for now.
|
| 1.2.2.7 | 08-Feb-2013 |
riz | Pull up following revision(s) (requested by christos in ticket #786): usr.sbin/makemandb/makemandb.c: revision 1.16 If you cannot parse .SH NAME, like in the case of the ksh93 man page where the .SH is followed by a conditional: .SH NAME .if \nZ=0 \{text text text .\} at least don't core-dump.
|
| 1.2.2.6 | 20-Nov-2012 |
riz | Pull up following revision(s) (requested by tron in ticket #676): usr.sbin/makemandb/apropos.c: revision 1.7 usr.sbin/makemandb/makemandb.8: revision 1.3 usr.sbin/makemandb/makemandb.8: revision 1.4 usr.sbin/makemandb/makemandb.8: revision 1.5 etc/rc.d/makemandb: revision 1.4 etc/weekly: revision 1.29 usr.sbin/makemandb/makemandb.c: revision 1.10 usr.sbin/makemandb/makemandb.c: revision 1.11 usr.sbin/makemandb/makemandb.c: revision 1.12 usr.sbin/makemandb/makemandb.c: revision 1.13 usr.sbin/makemandb/makemandb.c: revision 1.14 usr.sbin/makemandb/apropos.1: revision 1.5 etc/daily: revision 1.87 Add HISTORY section. From Abhinav Upadhyay in private mail. Sync with reality, From Abhinav Upadhyay in private mail. Fix typo in a message. Bug fix for PR 46733: > makemandb always reports the same number for "Total Number of new or > updated pages enountered" and "Total number of (hard or symbolic) > links found". Patch from Abhinav Upadhyay. Sort options in SYNOPSIS. Sync usage with manpage. Add -Q flag: Print only fatal error messages (i.e., when the database is left in an inconsistent state and needs manual intervention). From Abhinav Upadhyay <er.abhinav.upadhyay@gmail.com>. Use new makemandb -Q flag to be really quiet. Fix usage, from Abhinav Upadhyay <er.abhinav.upadhyay@gmail.com>. Use emalloc in one more place, like the rest of the code does. From Abhinav Upadhyay <er.abhinav.upadhyay@gmail.com>.
|
| 1.2.2.5 | 09-May-2012 |
riz | Pull up following revision(s) (requested by wiz in ticket #229): usr.sbin/makemandb/makemandb.c: revision 1.9 usr.sbin/makemandb/DBSCHEMA: revision 1.2 usr.sbin/makemandb/apropos-utils.c: revision 1.5 usr.sbin/makemandb/apropos-utils.h: revision 1.3 PR 46419 by Abhinav Upadhyay using his updated patch: Clean up after removing man page aliases.
|
| 1.2.2.4 | 07-May-2012 |
riz | Pull up following revision(s) (requested by wiz in ticket #228): usr.sbin/makemandb/makemandb.c: revision 1.8 The new apropos(1) incorrectly displays hyphens in the first line of the search results for a few man pages (for man(7) based man pages). Use patch from Abhinav Upadhyay in PR 46408 to fix this.
|
| 1.2.2.3 | 05-Mar-2012 |
sborrill | Pull up the following revisions(s) (requested by joerg in ticket #77): usr.sbin/makemandb/makemandb.c: revision 1.7
Fix inverted condition when handling stale entries.
|
| 1.2.2.2 | 02-Mar-2012 |
riz | Pull up following revision(s) (requested by joerg in ticket #70): usr.sbin/makemandb/makemandb.c: revision 1.6 Expand workaround for .so usage to do the chdir call just before starting parsing, not during the tree iteration. This gives it a chance to work.
|
| 1.2.2.1 | 18-Feb-2012 |
riz | Pull up following revision(s) (requested by joerg in ticket #4): usr.sbin/makemandb/apropos.c: revision 1.5 external/bsd/mdocml/dist/read.c: revision 1.6 external/bsd/mdocml/dist/mandoc.h: revision 1.2 usr.sbin/makemandb/makemandb.c: revision 1.3 usr.sbin/makemandb/makemandb.c: revision 1.4 usr.sbin/makemandb/makemandb.c: revision 1.5 usr.sbin/makemandb/Makefile: revision 1.2 etc/man.conf: revision 1.30 usr.sbin/makemandb/apropos.1: revision 1.3 external/bsd/mdocml/Makefile.inc: revision 1.15 Be a bit more friendly to man pages using the roff .so command by changing the current directory to the parent of the man -p entry, e.g. /usr/share/man for /usr/share/man1. Also handle hyphen replacement if it was used as plain input and no backslash sequence was used at all in the line. Add support for apropos -s like in the old apropos. Fix capitalisation. Add note about optional quotation. From Abhinav Upadhyay. Merge mandoc.h r1.99 and read.c 1.28 from upstream: Add mparse_readmem, which allows application to run the parser directly from memory, e.g. after de-compressing a document. Add support for compressed man pages in all the usual formats. Use mmap when dealing with plain files. Add entries for bzip2 and xz.
|
| 1.7.2.7 | 22-May-2014 |
yamt | sync with head.
for a reference, the tree before this commit was tagged as yamt-pagecache-tag8.
this commit was splitted into small chunks to avoid a limitation of cvs. ("Protocol error: too many arguments")
|
| 1.7.2.6 | 23-Jan-2013 |
yamt | sync with head
|
| 1.7.2.5 | 16-Jan-2013 |
yamt | sync with (a bit old) head
|
| 1.7.2.4 | 30-Oct-2012 |
yamt | sync with head
|
| 1.7.2.3 | 23-May-2012 |
yamt | sync with head.
|
| 1.7.2.2 | 17-Apr-2012 |
yamt | sync with head
|
| 1.7.2.1 | 02-Mar-2012 |
yamt | file makemandb.c was added on branch yamt-pagecache on 2012-04-17 00:09:49 +0000
|
| 1.14.2.4 | 20-Aug-2014 |
tls | Rebase to HEAD as of a few days ago.
|
| 1.14.2.3 | 23-Jun-2013 |
tls | resync from head
|
| 1.14.2.2 | 25-Feb-2013 |
tls | resync with head
|
| 1.14.2.1 | 20-Nov-2012 |
tls | Resync to 2012-11-19 00:00:00 UTC
|
| 1.22.2.1 | 10-Aug-2014 |
tls | Rebase.
|
| 1.24.2.3 | 14-Apr-2015 |
snj | Pull up following revision(s) (requested by christos in ticket #675): usr.sbin/makemandb/makemandb.c: revision 1.27 - handle section numbers that are not single digits - don't allocate and free needlessly
|
| 1.24.2.2 | 15-Mar-2015 |
snj | Pull up following revision(s) (requested by joerg in ticket #589): usr.sbin/makemandb/makemandb.c: revision 1.28 MDOC_MAX is a valid token if the type is text. Adjust.
|
| 1.24.2.1 | 07-Mar-2015 |
snj | Pull up following revision(s) (requested by skrll in ticket #564): usr.sbin/makemandb/makemandb.c: revision 1.26 Explicitly deal with end of lists. PR 49708.
|
| 1.39.2.4 | 26-Apr-2017 |
pgoyette | Sync with HEAD
|
| 1.39.2.3 | 07-Jan-2017 |
pgoyette | Sync with HEAD. (Note that most of these changes are simply $NetBSD$ tag issues.)
|
| 1.39.2.2 | 04-Nov-2016 |
pgoyette | Sync with HEAD
|
| 1.39.2.1 | 26-Jul-2016 |
pgoyette | Sync with HEAD
|
| 1.46.2.1 | 21-Apr-2017 |
bouyer | Sync with HEAD
|
| 1.47.2.2 | 11-May-2017 |
pgoyette | Sync with HEAD
|
| 1.47.2.1 | 02-May-2017 |
pgoyette | Sync with HEAD - tag prg-localcount2-base1
|
| 1.55.10.1 | 10-Jun-2019 |
christos | Sync with HEAD
|
| 1.55.8.1 | 06-Sep-2018 |
pgoyette | Sync with HEAD
Resolve a couple of conflicts (result of the uimin/uimax changes)
|
| 1.60.2.1 | 06-Jun-2022 |
martin | Pull up following revision(s) (requested by skrll in ticket #1465):
usr.sbin/makemandb/makemandb.c: revision 1.63
Don't index outside the mdocs array of function pointers. Analysis and suggested fixes from Tom Lane. I played it safe and went with (my variation of) the minimal fix.
PR port-hppa/56118: sporadic app crashes in HPPA -current
|
| 1.8 | 24-May-2023 |
lukem | obsolete mkstr(1) and xstr(1)
These were only used on PDP-11 for two programs we don't ship, and have been obsolete since the VAX days.
xstr never worked in the build.sh cross-build environment (22 years), or parallel make environment (nearly 28 years), didn't work in the orignal 386bsd import, and has never been needed in NetBSD as we don't have the older BSD programs (pascal, pre-nvi ex) that needed mkstr/xstr on PDP-11.
PR toolchain/35964
|
| 1.7 | 16-Apr-2021 |
msaitoh | - Remove obsolete cltom(9) and mtocl(9). - Add missing manual page's link of m_adj.
|
| 1.6 | 02-Jun-2020 |
kamil | Register ppoll
|
| 1.5 | 17-Jan-2020 |
maya | Remove uyurex(4).
This is a driver for a "nonsense machine" made by the art group Maywa-Denki in 2008. It was disabled by default.
Unfortunately even so it draws development attention (flaws found in the code, MP-ification needs) and it is best not to continue to maintain this driver.
Proposed without objections on tech-kern.
|
| 1.4 | 23-Dec-2019 |
maxv | Revert the removal of filemon.
|
| 1.3 | 18-Dec-2019 |
maxv | Retire filemon, discussed on tech-kern@.
|
| 1.2 | 27-Apr-2019 |
kamil | Add a C99 symbol to libm: nexttowardl
It's an alias for an already existing symbol nextafterl.
Patch obtained from <mgorny>
Detected by the LLVM buildbot breakage in tests.
|
| 1.1 | 18-Jun-2017 |
abhinav | branches: 1.1.6; Add a custom tokenizer which does not stem certain keywords.
Which keywords should not be stemmed is specified in the nostem.txt file. (Right now I have taken all the man page names, split them if they had underscores, removed common English words and converted everything to lowercase.)
The tokenizer itself is based on the Porter stemming tokenizer shipped with Sqlite. The code in custom_apropos_tokenizer.c is copy of that code with some modifications to prevent stemming keywords specified in nostem.txt.
Additionally, it now uses underscore `_' also as a token delimiter. Therefore, now it's possible to do query for `lwp' and all `_lwp_*' man page names will be matched. Or the query can be `unconst' and `__UNCONST' will be matched. This was not possible earlier, because underscore was not a delimiter and therefore the index would have __UNCONST as a key rather than UNCONST.
The tokenizer needs fts3_tokenizer.h file, which is not shipped with the amalgamation build of Sqlite, therefore it needs to be added here (unless we decide there is a better place for it).
To enforce using the new tokenizer, a schema version bump is needed
Since the tokenization is done both at the indexing time (via makemandb) and also while query time (via apropos or whatis), it will be needed to bump the schema version everytime nostem.txt is modified. Otherwise the index will consist of old tokens and desired changes will not be seen with apropos.
This should also fix the issue reported in PR bin/46255. Similar suggestion was also made on tech-userlevel@ recently: <http://mail-index.netbsd.org/tech-userlevel/2017/06/08/msg010620.html>
Thanks to christos@ for multiple rounds of reviews of the tokenizer code.
|
| 1.1.6.2 | 08-Apr-2020 |
martin | Merge changes from current as of 20200406
|
| 1.1.6.1 | 10-Jun-2019 |
christos | Sync with HEAD
|
| 1.6 | 24-Jun-2023 |
msaitoh | Fix typo in comment.
|
| 1.5 | 11-Sep-2022 |
gutteridge | makemandb/*: fix spelling of database and consistency of SQLite
|
| 1.4 | 24-May-2016 |
abhinav | Bring man pages in sync with reality.
Remove man pages run_query_html.3 and run_query_pager.3 as the corresponding functions have been removed from apropos-utils.c Ok by wiz@
|
| 1.3 | 18-Oct-2014 |
snj | src is too big these days to tolerate superfluous apostrophes. It's "its", people!
|
| 1.2 | 07-Feb-2012 |
wiz | branches: 1.2.4; No leading zeroes in date. New sentence, new line.
|
| 1.1 | 07-Feb-2012 |
joerg | Import the new apropos/whatis.
This code has been developed by Abhinav Upadhyay as part of Google's Summer of Code 2011. It uses libmandoc to parse man pages and builds a Full Text Index in a SQLite database. The combination of indexing the full manual page, filtering out stop words and ranking individual matches based on the section gives a much improved user experience.
The old makewhatis and friends are kept under MKMAKEMANDB=no for now.
|
| 1.2.4.2 | 17-Apr-2012 |
yamt | sync with head
|
| 1.2.4.1 | 07-Feb-2012 |
yamt | file run_query.3 was added on branch yamt-pagecache on 2012-04-17 00:09:49 +0000
|
| 1.3 | 24-May-2016 |
abhinav | Bring man pages in sync with reality.
Remove man pages run_query_html.3 and run_query_pager.3 as the corresponding functions have been removed from apropos-utils.c Ok by wiz@
|
| 1.2 | 07-Feb-2012 |
wiz | branches: 1.2.4; No leading zeroes in date.
|
| 1.1 | 07-Feb-2012 |
joerg | Import the new apropos/whatis.
This code has been developed by Abhinav Upadhyay as part of Google's Summer of Code 2011. It uses libmandoc to parse man pages and builds a Full Text Index in a SQLite database. The combination of indexing the full manual page, filtering out stop words and ranking individual matches based on the section gives a much improved user experience.
The old makewhatis and friends are kept under MKMAKEMANDB=no for now.
|
| 1.2.4.2 | 17-Apr-2012 |
yamt | sync with head
|
| 1.2.4.1 | 07-Feb-2012 |
yamt | file run_query_html.3 was added on branch yamt-pagecache on 2012-04-17 00:09:49 +0000
|
| 1.3 | 24-May-2016 |
abhinav | Bring man pages in sync with reality.
Remove man pages run_query_html.3 and run_query_pager.3 as the corresponding functions have been removed from apropos-utils.c Ok by wiz@
|
| 1.2 | 07-Feb-2012 |
wiz | branches: 1.2.4; No leading zeroes in date.
|
| 1.1 | 07-Feb-2012 |
joerg | Import the new apropos/whatis.
This code has been developed by Abhinav Upadhyay as part of Google's Summer of Code 2011. It uses libmandoc to parse man pages and builds a Full Text Index in a SQLite database. The combination of indexing the full manual page, filtering out stop words and ranking individual matches based on the section gives a much improved user experience.
The old makewhatis and friends are kept under MKMAKEMANDB=no for now.
|
| 1.2.4.2 | 17-Apr-2012 |
yamt | sync with head
|
| 1.2.4.1 | 07-Feb-2012 |
yamt | file run_query_pager.3 was added on branch yamt-pagecache on 2012-04-17 00:09:49 +0000
|
| 1.4 | 30-May-2016 |
abhinav | Add 'a' to the stopwords list. Ok from christos
|
| 1.3 | 31-Mar-2016 |
christos | update with the final list
|
| 1.2 | 31-Mar-2016 |
christos | PR/51018: Abhinav Upadhyay: Update stopwords list for apropos(1)
|
| 1.1 | 07-Feb-2012 |
joerg | branches: 1.1.4; Import the new apropos/whatis.
This code has been developed by Abhinav Upadhyay as part of Google's Summer of Code 2011. It uses libmandoc to parse man pages and builds a Full Text Index in a SQLite database. The combination of indexing the full manual page, filtering out stop words and ranking individual matches based on the section gives a much improved user experience.
The old makewhatis and friends are kept under MKMAKEMANDB=no for now.
|
| 1.1.4.2 | 17-Apr-2012 |
yamt | sync with head
|
| 1.1.4.1 | 07-Feb-2012 |
yamt | file stopwords.txt was added on branch yamt-pagecache on 2012-04-17 00:09:49 +0000
|
| 1.5 | 11-Sep-2022 |
gutteridge | makemandb/*: fix spelling of database and consistency of SQLite
|
| 1.4 | 16-Jun-2016 |
abhinav | Add man.conf(5) and man(1) in the SEE ALSO section.
|
| 1.3 | 22-May-2016 |
abhinav | Add options to whatis and apropos to accept custom man.conf.
makemandb(8), man(1) already use -C as an option to take man.conf path, so use the same option for whatis(1) and apropos(1) for consitency. apropos was using -C/-c to disable/enable context of the search matches, change that to -M/-m respectively.
|
| 1.2 | 06-Oct-2012 |
wiz | Make mandb path configurable. makemandb (and related tools) use the path from the _mandb variable from man.conf now.
Set _mandb in man.conf to same value as was used before.
From Abhinav Upadhyay <er.abhinav.upadhyay@gmail.com>.
|
| 1.1 | 07-Feb-2012 |
joerg | branches: 1.1.4; 1.1.8; Import the new apropos/whatis.
This code has been developed by Abhinav Upadhyay as part of Google's Summer of Code 2011. It uses libmandoc to parse man pages and builds a Full Text Index in a SQLite database. The combination of indexing the full manual page, filtering out stop words and ranking individual matches based on the section gives a much improved user experience.
The old makewhatis and friends are kept under MKMAKEMANDB=no for now.
|
| 1.1.8.1 | 20-Nov-2012 |
tls | Resync to 2012-11-19 00:00:00 UTC
|
| 1.1.4.3 | 30-Oct-2012 |
yamt | sync with head
|
| 1.1.4.2 | 17-Apr-2012 |
yamt | sync with head
|
| 1.1.4.1 | 07-Feb-2012 |
yamt | file whatis.1 was added on branch yamt-pagecache on 2012-04-17 00:09:50 +0000
|
| 1.10 | 03-Aug-2023 |
rin | makemandb: trailing whitespace
|
| 1.9 | 18-May-2019 |
leot | Reintroduce case insensitive comparison of name accidentally lost in last revision.
Discussed with <abhinav> via PR misc/54213, thanks!
|
| 1.8 | 18-May-2019 |
abhinav | PR misc/54213: Fix performance of whatis(1) when no matches are found
In revision 1.6 of whatis.c the query was modified to return matches for names found in MLINKS of the man pages as well. However it was slow. The reason probably being that it required a join. But more importantly the where condition on an FTS virtual table column is very slow. To avoid the join and the expensive where condition on the virtual table, add the name_desc column to the mandb_links table as well. This improves the performance of whatis(1) to the original level at the expense of slight data duplication.
Bump the schema to force database rebuild to take account for the new column addition
|
| 1.7 | 23-May-2017 |
abhinav | branches: 1.7.10; Make the name comparison case insensitive.
(The old whatis(1) also used to do case insensitive string comparisons).
|
| 1.6 | 23-Apr-2017 |
abhinav | Teach whatis(1) to handle MLINKS
Similar to apropos(1), whatis did not utilise the mandb_links table till now. Therefore, if it was asked about one of the links to a man page, it would error out. This change teaches whatis(1) to look up both the FTS table as well as the links table, thus ensuring that it is able to answer queries about MLINKS as well.
Comparision between outputs before this change and after this change:
#Before change $ whatis realloc realloc: not found
#after change $ ./whatis realloc realloc(3) - general memory allocation operations realloc(3) - general purpose memory allocation functions realloc(9) - general-purpose kernel memory allocator
|
| 1.5 | 22-May-2016 |
abhinav | branches: 1.5.2; Add options to whatis and apropos to accept custom man.conf.
makemandb(8), man(1) already use -C as an option to take man.conf path, so use the same option for whatis(1) and apropos(1) for consitency. apropos was using -C/-c to disable/enable context of the search matches, change that to -M/-m respectively.
|
| 1.4 | 06-Oct-2012 |
wiz | Make mandb path configurable. makemandb (and related tools) use the path from the _mandb variable from man.conf now.
Set _mandb in man.conf to same value as was used before.
From Abhinav Upadhyay <er.abhinav.upadhyay@gmail.com>.
|
| 1.3 | 20-Feb-2012 |
joerg | branches: 1.3.2; 1.3.4; SQLite doesn't use the FTS index for equal ops, so force it to do a FTS search first. Drops run time by a factor of 6 for "whatis man". Found by Abhinav Upadhyay.
|
| 1.2 | 07-Feb-2012 |
joerg | branches: 1.2.2; Fix C&P error with $NetBSD$
|
| 1.1 | 07-Feb-2012 |
joerg | Import the new apropos/whatis.
This code has been developed by Abhinav Upadhyay as part of Google's Summer of Code 2011. It uses libmandoc to parse man pages and builds a Full Text Index in a SQLite database. The combination of indexing the full manual page, filtering out stop words and ranking individual matches based on the section gives a much improved user experience.
The old makewhatis and friends are kept under MKMAKEMANDB=no for now.
|
| 1.2.2.1 | 20-Feb-2012 |
sborrill | Pull up the following revisions(s) (requested by joerg in ticket #15): usr.sbin/makemandb/apropos.1: revision 1.4 usr.sbin/makemandb/makemandb.8: revision 1.2 usr.sbin/makemandb/whatis.c: revision 1.3
Add reference to whatis(1). SQLite doesn't use the FTS index for equal ops, so force it to do a FTS search first. Drops run time by a factor of 6 for "whatis man".
|
| 1.3.4.1 | 20-Nov-2012 |
tls | Resync to 2012-11-19 00:00:00 UTC
|
| 1.3.2.3 | 30-Oct-2012 |
yamt | sync with head
|
| 1.3.2.2 | 17-Apr-2012 |
yamt | sync with head
|
| 1.3.2.1 | 20-Feb-2012 |
yamt | file whatis.c was added on branch yamt-pagecache on 2012-04-17 00:09:50 +0000
|
| 1.5.2.1 | 26-Apr-2017 |
pgoyette | Sync with HEAD
|
| 1.7.10.1 | 10-Jun-2019 |
christos | Sync with HEAD
|