[Tools-discuss] all_id2.txt not getting updated

Post by Eric Rescorla
On a recent rsync, all_id2.txt shows draft-10 of a draft, but -11 is in the
repo.
08:05:11|~/dev/ietf-phab (master)$ grep ietf-ipsecme-split-dns
ids/all_id2.txt

On the IETF production server, I have:

ietfa:~ietf-ftp/internet-drafts # l all_id2.txt
-rw-rw-rw- 2 wwwrun www 7483796 Jul 19 08:07 all_id2.txt

ietfa:~ietf-ftp/internet-drafts # l *ietf-ipsecme-split-dns*
-rw-rw-rw- 2 wwwrun www 29620 Jul 19 07:46 draft-ietf-ipsecme-split-dns-11.txt
-rw-rw-rw- 2 wwwrun www 29310 Jul 19 07:46 draft-ietf-ipsecme-split-dns-11.xml

ietfa:~iietf-ftp/internet-drafts # grep ietf-ipsecme-split-dns all_id2.txt
draft-ietf-ipsecme-split-dns-11 -1 Active AD Evaluation::AD
Followup 2018-07-19 ipsecme sec Eric Rescorla
Proposed Standard .txt,.xmlSplit DNS Configuration for
IKEv2 Tommy Pauly <***@apple.com>, Paul Wouters
<***@redhat.com> David Waltermire
<***@nist.gov> Eric Rescorla <***@rtfm.com>
draft-pauly-ipsecme-split-dns-02 -1 Replaced
draft-ietf-ipsecme-split-dns 2016-09-21 ipsecme sec
Split DNS Configuration for IKEv2
Tommy Pauly <***@apple.com>, Paul Wouters <***@redhat.com>

Glen

___________________________________________________________
Tools-discuss mailing list
Tools-***@ietf.org
https://www.ietf.org/mailman/listinfo/tools-discuss

Please report datatracker.ietf.org and mailarchive.ietf.org
bugs at http://tools.ietf.org/tools/ietfdb
or send email to datatracker-***@ietf.org

Please report tools.ietf.org bugs at
http://tools.ietf.org/tools/issues
or send email to ***@tools.ietf.org

Eric Rescorla

2018-07-19 15:22:05 UTC

Post by Eric Rescorla
On a recent rsync, all_id2.txt shows draft-10 of a draft, but -11 is in

the

Post by Eric Rescorla
repo.
08:05:11|~/dev/ietf-phab (master)$ grep ietf-ipsecme-split-dns
ids/all_id2.txt

ietfa:~ietf-ftp/internet-drafts # l all_id2.txt
-rw-rw-rw- 2 wwwrun www 7483796 Jul 19 08:07 all_id2.txt

Yes, it is updated now. However, it seems that it's possible for it to get
temporarily out of sync, which is suboptimal.

-Ekr

Glen

2018-07-19 15:32:07 UTC

Post by Eric Rescorla
Yes, it is updated now. However, it seems that it's possible for it to get
temporarily out of sync, which is suboptimal.

Agreed. IIRC all_id2.txt is generated on a periodic cron job, so it
may at any time be momentarily out of sync... up to, I suppose, 60
minutes.

I don't think regenerating a large file like that on every draft
upload is... workable? I must leave it to others to make that call,
and/or to determine if there is a better way to get the data you need
in real time (API Datatracker call?)

Glen

___________________________________________________________
Tools-discuss mailing list
Tools-***@ietf.org
https://www.ietf.org/mailman/listinfo/tools-discuss

Please report datatracker.ietf.org and mailarchive.ietf.org
bugs at http://tools.ietf.org/tools/ietfdb
or send email to datatracker-***@ietf.org

Please report tools.ietf.org bugs at
http://tools.ietf.org/tools/issues
or send email to ***@tools.ietf.org

Martin Thomson

2018-07-19 15:37:38 UTC

Post by Glen
I don't think regenerating a large file like that on every draft
upload is... workable? I must leave it to others to make that call,
and/or to determine if there is a better way to get the data you need
in real time (API Datatracker call?)

I would greatly appreciate having an API. I have recently found a
need to retrieve draft status. Separately, we've had requests for an
API to get working group metadata (chair identities, for instance).

___________________________________________________________
Tools-discuss mailing list
Tools-***@ietf.org
https://www.ietf.org/mailman/listinfo/tools-discuss

Please report datatracker.ietf.org and mailarchive.ietf.org
bugs at http://tools.ietf.org/tools/ietfdb
or send email to datatracker-***@ietf.org

Please report tools.ietf.org bugs at
http://tools.ietf.org/tools/issues
or send email to ***@tools.ietf.org

Henrik Levkowetz

2018-07-19 15:42:40 UTC

Henrik Levkowetz

2018-07-19 15:47:16 UTC

Oh, and you can force json by adding a 'format=json' query argument.
Otherwise will choose between returning xml or json depending your
http accept header field.

Henrik

Post by Henrik Levkowetz

https://datatracker.ietf.org/api/
https://datatracker.ietf.org/api/v1/
https://datatracker.ietf.org/api/v1/group/role/?group__acronym=httpbis
https://datatracker.ietf.org/api/v1/doc/document/?name=draft-ietf-ipsecme-split-dns
Henrik
___________________________________________________________
Tools-discuss mailing list
https://www.ietf.org/mailman/listinfo/tools-discuss
Please report datatracker.ietf.org and mailarchive.ietf.org
bugs at http://tools.ietf.org/tools/ietfdb
Please report tools.ietf.org bugs at
http://tools.ietf.org/tools/issues

Martin Thomson

2018-07-19 15:52:35 UTC

Oh, that's nice. Thanks.

I apologize for having opened tickets for these.

Post by Henrik Levkowetz

___________________________________________________________
Tools-discuss mailing list
Tools-***@ietf.org
https://www.ietf.org/mailman/listinfo/tools-discuss

Please report datatracker.ietf.org and mailarchive.ietf.org
bugs at http://tools.ietf.org/tools/ietfdb
or send email to datatracker-***@ietf.org

Please report tools.ietf.org bugs at
http://tools.ietf.org/tools/issues
or send email to ***@tools.ietf.org

Henrik Levkowetz

2018-07-19 15:56:13 UTC

Hi Martin,

Post by Martin Thomson
Oh, that's nice. Thanks.
I apologize for having opened tickets for these.

No worries.

Henrik

Post by Henrik Levkowetz

___________________________________________________________
Tools-discuss mailing list
https://www.ietf.org/mailman/listinfo/tools-discuss
Please report datatracker.ietf.org and mailarchive.ietf.org
bugs at http://tools.ietf.org/tools/ietfdb
Please report tools.ietf.org bugs at
http://tools.ietf.org/tools/issues

Eric Rescorla

2018-07-19 15:51:29 UTC

Post by Eric Rescorla
Yes, it is updated now. However, it seems that it's possible for it to

get

Post by Eric Rescorla
temporarily out of sync, which is suboptimal.

Really? It's order 30K lines. This seems like it ought to be very
straightforward.
For reference, the time for me to retrieve it and parse it in my script is
<2s.

I must leave it to others to make that call,

Post by Glen
and/or to determine if there is a better way to get the data you need
in real time (API Datatracker call?)

Perhaps. What I need is "here are the new drafts since the following time",
so i do that by taking the entire draft DB and just looking for new stuff. I
imagine we could invent some kind of timestamp/checkpoint protocol,
but that seems pretty complicated and brittle.

-Ekr

Chris Morrow

2018-07-19 17:37:22 UTC

On Thu, 19 Jul 2018 11:51:29 -0400,

[1 <multipart/alternative (7bit)>]
[1.1 <text/plain; UTF-8 (7bit)>]

Post by Glen
I don't think regenerating a large file like that on every draft
upload is... workable?

Really? It's order 30K lines. This seems like it ought to be very
straightforward.
For reference, the time for me to retrieve it and parse it in my script is
<2s.

it MIGHT be that you have to walk the directory tree to build the
content for the file, right? and depending upon filesystem with ~30k
files that could be an unhappy operation.

-chris

___________________________________________________________
Tools-discuss mailing list
Tools-***@ietf.org
https://www.ietf.org/mailman/listinfo/tools-discuss

Please report datatracker.ietf.org and mailarchive.ietf.org
bugs at http://tools.ietf.org/tools/ietfdb
or send email to datatracker-***@ietf.org

Please report tools.ietf.org bugs at
http://tools.ietf.org/tools/issues
or send email to ***@tools.ietf.org

Glen

2018-07-19 17:48:43 UTC

Post by Eric Rescorla
Really? It's order 30K lines. This seems like it ought to be very
straightforward.
For reference, the time for me to retrieve it and parse it in my script is
<2s.

it MIGHT be that you have to walk the directory tree to build the
content for the file, right? and depending upon filesystem with ~30k
files that could be an unhappy operation.

Correct on every count. The size of the file is not a valid indicator
of processing burden.

And just to be clear, although I'm sure many people know this... my
role at AMS for the IETF is to handle operations (keep the servers
running, clean, updated, and answer what questions I can) but the
actual engineering of things (such as the IETF's Datatracker software,
and related things developed by the IETF) is the TMC's area, and is
handled by Henrik and others.

So I was trying to provide a snapshot of what I saw to help out here,
but I don't know the methods used to generate that file, and so could
not even comment reliably on it. (Saying this just so you know I'm
not ignoring - I simply don't have the answer.)

Glen

___________________________________________________________
Tools-discuss mailing list
Tools-***@ietf.org
https://www.ietf.org/mailman/listinfo/tools-discuss

Please report datatracker.ietf.org and mailarchive.ietf.org
bugs at http://tools.ietf.org/tools/ietfdb
or send email to datatracker-***@ietf.org

Please report tools.ietf.org bugs at
http://tools.ietf.org/tools/issues
or send email to ***@tools.ietf.org

Eric Rescorla

2018-07-19 18:21:03 UTC

Post by Eric Rescorla
Really? It's order 30K lines. This seems like it ought to be very
straightforward.
For reference, the time for me to retrieve it and parse it in my script

Post by Eric Rescorla
<2s.

it MIGHT be that you have to walk the directory tree to build the
content for the file, right? and depending upon filesystem with ~30k
files that could be an unhappy operation.

Correct on every count. The size of the file is not a valid indicator
of processing burden.

Well, I suppose that's true as a general matter (e.g., if it contains a
giant RSA key which
I'm expected to factor), but this kind of text processing task is typically
pretty linear.

I don't know how the data is stored on the server, but it's usually pretty
straightforward
to build a cache that lets you generate this kind of data quickly. With
that said, scraping
the filesystem should also be pretty fast. I just measured this quickly on
my laptop
with an rsynced copy and building up the list of drafts took less than a
second. It
might take a little longer if you had to extract the metadata, but again,
this kind of problem
can be addressed with caching.

Post by Glen
And just to be clear, although I'm sure many people know this... my
role at AMS for the IETF is to handle operations (keep the servers
running, clean, updated, and answer what questions I can) but the
actual engineering of things (such as the IETF's Datatracker software,
and related things developed by the IETF) is the TMC's area, and is
handled by Henrik and others.
So I was trying to provide a snapshot of what I saw to help out here,
but I don't know the methods used to generate that file, and so could
not even comment reliably on it. (Saying this just so you know I'm
not ignoring - I simply don't have the answer.)

Sure. Thanks for your response.

-Ekr

Post by Glen
Glen

Robert Sparks

2018-07-19 18:37:01 UTC

Given what you're doing, would a 'new-revision' document feed be useful?

On Thu, Jul 19, 2018 at 10:37 AM, Chris Morrow

Post by Eric Rescorla
Really? It's order 30K lines. This seems like it ought to be very
straightforward.
For reference, the time for me to retrieve it and parse it in

my script is

Post by Eric Rescorla
<2s.

it MIGHT be that you have to walk the directory tree to build the
content for the file, right? and depending upon filesystem with ~30k
files that could be an unhappy operation.

Correct on every count.Â The size of the file is not a valid indicator
of processing burden.
Well, I suppose that's true as a general matter (e.g., if it contains
a giant RSA key which
I'm expected to factor), but this kind of text processing task is
typically pretty linear.
I don't know how the data is stored on the server, but it's usually
pretty straightforward
to build a cache that lets you generate this kind of data quickly.
With that said, scraping
the filesystem should also be pretty fast. I just measured this
quickly on my laptop
with an rsynced copy and building up the list of drafts took less than
a second. It
might take a little longer if you had to extract the metadata, but
again, this kind of problem
can be addressed with caching.
And just to be clear, although I'm sure many people know this...Â my
role at AMS for the IETF is to handle operations (keep the servers
running, clean, updated, and answer what questions I can) but the
actual engineering of things (such as the IETF's Datatracker software,
and related things developed by the IETF) is the TMC's area, and is
handled by Henrik and others.
So I was trying to provide a snapshot of what I saw to help out here,
but I don't know the methods used to generate that file, and so could
not even comment reliably on it.Â Â (Saying this just so you know I'm
not ignoring - I simply don't have the answer.)
Sure. Thanks for your response.
-Ekr
Glen
___________________________________________________________
Tools-discuss mailing list
https://www.ietf.org/mailman/listinfo/tools-discuss
Please report datatracker.ietf.org and mailarchive.ietf.org
bugs at http://tools.ietf.org/tools/ietfdb
Please report tools.ietf.org bugs at
http://tools.ietf.org/tools/issues

Eric Rescorla

2018-07-19 18:39:14 UTC

Post by Robert Sparks
Given what you're doing, would a 'new-revision' document feed be useful?

Probably would have, but I ended up just using the directory listing.

-Ekr

Post by Robert Sparks

Post by Eric Rescorla
Really? It's order 30K lines. This seems like it ought to be very
straightforward.
For reference, the time for me to retrieve it and parse it in my

script is

Post by Eric Rescorla
<2s.

it MIGHT be that you have to walk the directory tree to build the
content for the file, right? and depending upon filesystem with ~30k
files that could be an unhappy operation.

Correct on every count. The size of the file is not a valid indicator
of processing burden.

Well, I suppose that's true as a general matter (e.g., if it contains a
giant RSA key which
I'm expected to factor), but this kind of text processing task is
typically pretty linear.
I don't know how the data is stored on the server, but it's usually pretty
straightforward
to build a cache that lets you generate this kind of data quickly. With
that said, scraping
the filesystem should also be pretty fast. I just measured this quickly on
my laptop
with an rsynced copy and building up the list of drafts took less than a
second. It
might take a little longer if you had to extract the metadata, but again,
this kind of problem
can be addressed with caching.

Post by Eric Rescorla
And just to be clear, although I'm sure many people know this... my
role at AMS for the IETF is to handle operations (keep the servers
running, clean, updated, and answer what questions I can) but the
actual engineering of things (such as the IETF's Datatracker software,
and related things developed by the IETF) is the TMC's area, and is
handled by Henrik and others.
So I was trying to provide a snapshot of what I saw to help out here,
but I don't know the methods used to generate that file, and so could
not even comment reliably on it. (Saying this just so you know I'm
not ignoring - I simply don't have the answer.)

Sure. Thanks for your response.
-Ekr

Post by Eric Rescorla
Glen

___________________________________________________________
Please report datatracker.ietf.org and mailarchive.ietf.org
bugs at http://tools.ietf.org/tools/ietfdb
Please report tools.ietf.org bugs athttp://tools.ietf.org/tools/issues

Chris Morrow

2018-07-19 21:42:56 UTC

On Thu, 19 Jul 2018 14:21:03 -0400,

Post by Eric Rescorla
I don't know how the data is stored on the server, but it's usually pretty
straightforward
to build a cache that lets you generate this kind of data quickly. With
that said, scraping
the filesystem should also be pretty fast. I just measured this quickly on
my laptop
with an rsynced copy and building up the list of drafts took less than a
second. It
might take a little longer if you had to extract the metadata, but again,
this kind of problem
can be addressed with caching.

'depending on the file system'

with, for example, ext3 30k files can be very expensive.

(I don't know what the server is, but tools == debian and datatracker
appears to be fbsd, so you can make some guesses about filesystem
type)

-chris

___________________________________________________________
Tools-discuss mailing list
Tools-***@ietf.org
https://www.ietf.org/mailman/listinfo/tools-discuss

Please report datatracker.ietf.org and mailarchive.ietf.org
bugs at http://tools.ietf.org/tools/ietfdb
or send email to datatracker-***@ietf.org

Please report tools.ietf.org bugs at
http://tools.ietf.org/tools/issues
or send email to ***@tools.ietf.org

Glen

2018-07-19 22:03:51 UTC

Post by Chris Morrow
(I don't know what the server is, but tools == debian and datatracker
appears to be fbsd, so you can make some guesses about filesystem
type)

The production servers run OpenSuse Leap 42.3, the filesystem is
currently ext4, which was the newest filesystem when we took these
over years ago. I know better filesystems options exist now, and we
may consider a migration when we do a future server upgrade.

But to clarify - I did not mean to imply that we could not cope with a
given frequency of transaction - the servers are quite powerful and
could probably "cope" with a bit more than they currently do (although
they do quite a bit already.) I was clearly imprecise, and my
imprecision caused a follow-on discussion, for which I apologize.
(Even though we all love such discussions anyway. :-)

Rather, I think I meant to say something like:

At present, the regeneration of all_id2.txt (and similar files) is
done by the Datatracker, in a job scheduled by cron to run once an
hour. I do not know how feasible it would be to - for example -
change it to run the job once per draft submission instead of once per
hour. I have no idea whether it would be helpful or off point,
whether it would be right or wrong, better or worse, but, absent such
a change, the original assertion that the file can be out of immediate
sync at any given instant seems to me to be correct. These are all
questions for Henrik and Engineering volunteers.

And while I'm correcting myself, I was reminded that another statement
I made was also imprecise. Instead of TMC, I should have said "Tools
Team". They are the ones who handle the engineering and new code
development, with Henrik being a primary point of contact for topics
like this.

Hope this is a bit more clear.

Glen

___________________________________________________________
Tools-discuss mailing list
Tools-***@ietf.org
https://www.ietf.org/mailman/listinfo/tools-discuss

Please report datatracker.ietf.org and mailarchive.ietf.org
bugs at http://tools.ietf.org/tools/ietfdb
or send email to datatracker-***@ietf.org

Please report tools.ietf.org bugs at
http://tools.ietf.org/tools/issues
or send email to ***@tools.ietf.org

Eric Rescorla

2018-07-21 14:33:22 UTC

Post by Chris Morrow
(I don't know what the server is, but tools == debian and datatracker
appears to be fbsd, so you can make some guesses about filesystem
type)

Do you happen to know the name of the script? That would save me some time
trawling through the tracker code.

-Ekr

I do not know how feasible it would be to - for example -

Post by Glen
change it to run the job once per draft submission instead of once per
hour. I have no idea whether it would be helpful or off point,
whether it would be right or wrong, better or worse, but, absent such
a change, the original assertion that the file can be out of immediate
sync at any given instant seems to me to be correct. These are all
questions for Henrik and Engineering volunteers.
And while I'm correcting myself, I was reminded that another statement
I made was also imprecise. Instead of TMC, I should have said "Tools
Team". They are the ones who handle the engineering and new code
development, with Henrik being a primary point of contact for topics
like this.
Hope this is a bit more clear.
Glen
___________________________________________________________
Tools-discuss mailing list
https://www.ietf.org/mailman/listinfo/tools-discuss
Please report datatracker.ietf.org and mailarchive.ietf.org
bugs at http://tools.ietf.org/tools/ietfdb
Please report tools.ietf.org bugs at
http://tools.ietf.org/tools/issues

Chris Morrow

2018-07-21 17:49:35 UTC

On Sat, 21 Jul 2018 10:33:22 -0400,

[1 <multipart/alternative (7bit)>]
[1.1 <text/plain; UTF-8 (7bit)>]

Post by Chris Morrow
(I don't know what the server is, but tools == debian and datatracker
appears to be fbsd, so you can make some guesses about filesystem
type)

Do you happen to know the name of the script? That would save me some time
trawling through the tracker code.

https://github.com/mcr/ietfdb/blob/210f761f678600cd81d8824ade2be649451aeda7/ietf/idindex/index.py

may be relevant to your interests. though this appears to pull all ids from 'some data store' convert to txt and update the id_blah file when done.

-Ekr
I do not know how feasible it would be to - for example -

[1.2 <text/html; UTF-8 (quoted-printable)>]
[2 <text/plain; us-ascii (7bit)>]
___________________________________________________________
Tools-discuss mailing list
https://www.ietf.org/mailman/listinfo/tools-discuss
Please report datatracker.ietf.org and mailarchive.ietf.org
bugs at http://tools.ietf.org/tools/ietfdb
Please report tools.ietf.org bugs at
http://tools.ietf.org/tools/issues

Michael Richardson

2018-07-21 22:06:09 UTC

Post by Glen
At present, the regeneration of all_id2.txt (and similar files) is
done by the Datatracker, in a job scheduled by cron to run once an
hour.

Do you happen to know the name of the script? That would save me some time
trawling through the tracker code.

https://github.com/mcr/ietfdb/blob/210f761f678600cd81d8824ade2be649451aeda7/ietf/idindex/index.py

That tree is 8 years old.
I shall remove it.

--
] Never tell me the odds! | ipv6 mesh networks [
] Michael Richardson, Sandelman Software Works | network architect [
] ***@sandelman.ca http://www.sandelman.ca/ | ruby on rails [

___________________________________________________________
Tools-discuss mailing list
Tools-***@ietf.org
https://www.ietf.org/mailman/listinfo/tools-discuss

Please report datatracker.ietf.org and mailarchive.ietf.org
bugs at http://tools.ietf.org/tools/ietfdb
or send email to datatracker-***@ietf.org

Please report tools.ietf.org bugs at
http://tools.ietf.org/tools/issues
or send email to ***@tools.ietf.org

Glen

2018-07-21 17:59:11 UTC

Hi Eric (and thanks Chris for your email also!)

Post by Glen
At present, the regeneration of all_id2.txt (and similar files) is
done by the Datatracker, in a job scheduled by cron to run once an
hour.

Do you happen to know the name of the script? That would save me some time
trawling through the tracker code.

Absolutely. Sort of. :-) In the old days we inherited just a bunch
of random scripts. I tried consolidating them over the years, but
then Henrik took a much greater step in building them right into the
datatracker. So we have this crontab that runs:

PATH=/sbin:/usr/sbin:/bin:/usr/bin
DTDIR=/a/www/ietf-datatracker/web
@monthly wwwrun test -x $DTDIR/bin/monthly && $DTDIR/bin/monthly
@weekly wwwrun test -x $DTDIR/bin/weekly && $DTDIR/bin/weekly
05 00 * * * wwwrun test -x $DTDIR/bin/daily && $DTDIR/bin/daily
05 1-23 * * * wwwrun test -x $DTDIR/bin/hourly && $DTDIR/bin/hourly
05 * * * * mailman test -x $DTDIR/bin/mm_hourly && $DTDIR/bin/mm_hourly

And I'm guessing it's /bin/hourly that you want.

There are still some legacy scripts that *also* run at 5 past the hour
- I don't *think* they're relevant but I'll mention them for
completeness...

PYTHONPATH=/a/www/ietf-datatracker/web export PYTHONPATH
/a/www/ietf-datatracker/web/ietf/bin/rfc-editor-index-updates #
(replaces mirror_rfc_index)
/a/www/ietf-datatracker/web/ietf/bin/rfc-editor-queue-updates #
(replaces mirror_rfc_queue)
/a/www/ietf-datatracker/web/ietf/bin/generate-draft-aliases && \
( cd /a/postfix; /usr/sbin/postalias -o draft-aliases; ) && \
( cd /a/postfix; /usr/sbin/postmap -o draft-virtual; )
/a/www/ietf-datatracker/web/ietf/bin/generate-wg-aliases && \
( cd /a/postfix; /usr/sbin/postalias -o group-aliases; ) && \
( cd /a/postfix; /usr/sbin/postmap -o group-virtual; )

Hope this helps!

Glen

___________________________________________________________
Tools-discuss mailing list
Tools-***@ietf.org
https://www.ietf.org/mailman/listinfo/tools-discuss

Please report datatracker.ietf.org and mailarchive.ietf.org
bugs at http://tools.ietf.org/tools/ietfdb
or send email to datatracker-***@ietf.org

Please report tools.ietf.org bugs at
http://tools.ietf.org/tools/issues
or send email to ***@tools.ietf.org

Eric Rescorla

2018-07-21 18:17:15 UTC

Thanks!

Post by Glen
Hi Eric (and thanks Chris for your email also!)

Post by Glen
At present, the regeneration of all_id2.txt (and similar files) is
done by the Datatracker, in a job scheduled by cron to run once an
hour.

Do you happen to know the name of the script? That would save me some

time

Post by Eric Rescorla
trawling through the tracker code.

Absolutely. Sort of. :-) In the old days we inherited just a bunch
of random scripts. I tried consolidating them over the years, but
then Henrik took a much greater step in building them right into the
PATH=/sbin:/usr/sbin:/bin:/usr/bin
DTDIR=/a/www/ietf-datatracker/web
@monthly wwwrun test -x $DTDIR/bin/monthly && $DTDIR/bin/monthly
@weekly wwwrun test -x $DTDIR/bin/weekly && $DTDIR/bin/weekly
05 00 * * * wwwrun test -x $DTDIR/bin/daily && $DTDIR/bin/daily
05 1-23 * * * wwwrun test -x $DTDIR/bin/hourly && $DTDIR/bin/hourly
05 * * * * mailman test -x $DTDIR/bin/mm_hourly &&
$DTDIR/bin/mm_hourly
And I'm guessing it's /bin/hourly that you want.
There are still some legacy scripts that *also* run at 5 past the hour
- I don't *think* they're relevant but I'll mention them for
completeness...
PYTHONPATH=/a/www/ietf-datatracker/web export PYTHONPATH
/a/www/ietf-datatracker/web/ietf/bin/rfc-editor-index-updates #
(replaces mirror_rfc_index)
/a/www/ietf-datatracker/web/ietf/bin/rfc-editor-queue-updates #
(replaces mirror_rfc_queue)
/a/www/ietf-datatracker/web/ietf/bin/generate-draft-aliases && \
( cd /a/postfix; /usr/sbin/postalias -o draft-aliases; ) && \
( cd /a/postfix; /usr/sbin/postmap -o draft-virtual; )
/a/www/ietf-datatracker/web/ietf/bin/generate-wg-aliases && \
( cd /a/postfix; /usr/sbin/postalias -o group-aliases; ) && \
( cd /a/postfix; /usr/sbin/postmap -o group-virtual; )
Hope this helps!
Glen

Robert Sparks

2018-07-23 02:58:52 UTC

The essential bit you're looking for is here:

https://trac.tools.ietf.org/tools/ietfdb/browser/trunk/ietf/idindex/index.py#L102

Further up the file, you'll find what builds all_id.txt as well.

Cron calls this thing periodically:

https://trac.tools.ietf.org/tools/ietfdb/browser/trunk/ietf/idindex/generate_all_id2_txt.py

Which is a thin wrapper around the first thing I point to above.

(If you really need the glue from cron, this file is run hourly:

https://trac.tools.ietf.org/tools/ietfdb/browser/trunk/bin/hourly

)

RjS

Post by Eric Rescorla
Thanks!
Hi Eric (and thanks Chris for your email also!)