|
Climate Prediction Center Data: Meteorological Data Servers
|
|
Climate, Weather Analyses, Weather Forecasts
|
| |
|
Please choose a server:
|
|
|
Part of the AMS 2010 short course on climate datasets:
ftp://ftp.cpc.ncep.noaa.gov/wd51we/ams2010_sc/. This section
has information on the NCEP Reanalyses and how to obtain and use them.
Also has hints on how to obtain data from the NOMAD servers.
|
Suggestions
|
-
Don't bookmark or post URL's of plots that you have created
because the plots are automatically deleted after 20 minutes.
|
|
|
|
|
|
|
-
For Nomad questions/comments: Jordan.Alpert@noaa.gov
|
|
|
|
|
Programming Suggestions
|
Generated plots are periodically deleted to recover space.
After you make a plot, don't wait to download the plot.
|
Check the URLs that your program generates.
valid months: jan feb mar apr may jun jul aug sep oct nov dec
|
|
Try to generate plots sequentially not 400 at a time.
|
|
|
FTP2U Warning
|
|
As of 1/2005, FTP2U will be replaced by a new method to download data. This
method is (1) faster, (2) consumes fewer server resources, (3) super easy to
automate/script, (4) doesn't require server scripts and (5) makes our life easier.
You are strongly urged to change to the new method.
|
|
|
Fair Use/System Busy:
|
|
2/2007: We all want to get our work done as quickly as possible. Nothing
wrong with that. However, sending 100 jobs a second actually
causes the machine to get less done. Asking if the
forecast is ready (10 * second * 30 files * 10 users ) puts a
severe load on the system. We've had many cases where a flood of
file queries caused all the web applications to shut down. So we
started using the firewall to deny access to IP addresses. So
if get too many errors, submit too many jobs/minute, submit too many
resouce intensive jobs/minute, suck up all the bandwidth, your IP
address may be turned off at the firewall. If you go over some
limit, the firewall rejects your IP address for periods, of 20 seconds,
60 seconds, 8 minutes, a day and even forever. The penalty has a degree
of randomness just like real life. Triggering the time out depends on
system load and we also periodically retune the parameters. So don't
expect a script that "floods" the system today will work next week.
|
|
|
Network Problems or Bad Configuration: User Reports
|
|
If the files are not up to date, contact the person(s) responsible for the data set.
|
|
If most of the files are missing, contact the person(s) responsible and me
(Wesley.Ebisuzaki@noaa.gov) as someone has probably moved, renamed, deleted the files.
|
|
If your browser says that Server 1 or 2 will not respond, we either have a broken
network, workstation or web server. Not much you can do about network problems.
By the time the email gets through, the problem is fixed. As for a dead
workstation, that gets noticed pretty quickly during business hours. For
the non-business hours, the workstation people do not work 24x7. Now for
a dead web server. try another server. If the server is really dead, a
cron job will try to restart the server every few hours. Doesn't hurt complaining
to Jordan.Alpert@noaa.gov or Wesley.Ebisuzaki@noaa.gov.
|
|
The most frequent problem is when the software complains about a "Network or
Configuration Problem". This is usually caused by a stale NFS handle. You
can check for this problem by checking a number of similar datasets. Send
a message to Jordan.Alpert@noaa.gov or Wesley.Ebisuzaki@noaa.gov and we
forward the message to the system people to reboot the machine.
|
|
For problems with server3, contact Wesley.Ebisuzaki@noaa.gov.
|
|
|
Making Plots and Downloading Data Automatically
|
|
Plotting with the browser is fun for the first 10,000 plots. However, after the
10,001 plot you may wonder whether there is a better way. Well, there is.
Get a copy of wwwgrab
and put your web requests into a cron or batch job. There are
examples
for downloading gdas and mrf plots. Before you go wild with making plots,
remember that each plot still takes that same time to generate. By the
way, wwwgrab has been used to automate downloading data from the FTP2U
server (see example).
|
|
If you routinely make prints of the plots, you should consider using wwwgrab.
You can automate the downloading and printing of the postscript version
of the plots. The postscript version is so much sharper than the png. No
examples are provided but it is so easy that even I can do it.
|
|
Note:
Selectable color range - set white=0 for anomaly plots
You can download
raw time series data into your spreadsheets.
|
|
Animation
is available. The animation facility, however, is not for the populace.
It requires (1) X windows, (2) a fast connection (56K modems are not fast)
and (3) a real TCP/IP address (people behind a firewall need not apply).
Use xkill to stop an animation.
|
|
Last Resort for firewall and slow FTP PROBLEMS.
|
|
Font too small? Run your display at 800x600, use IE and
make your local web designer happy.
|
|
comments: Wesley.Ebisuzaki@noaa.gov
|
www.nws.noaa.gov 
