I would like to do something that may be ...

[Gawdzilla Sama]

...impossible.

I want to merge a bunch of html files together so the file can be searched. The pages look like this: http://www.history.navy.mil/photos/imag ... 01395c.htm

Any suggestions?

Any suggestions that are sane?

[Tyrannical]

Doesn't google offer a search widget for indexing web sites?

[Thinking Aloud]

Agh. Looks like you've got yourself something that was started in the days before databases. Short of using a spider-search as suggested above, I'm not sure there's a quick solution.

For a decent custom search you'd need to somehow import the data from each page as a record into a db and use that to serve pages dynamically. But I suspect that's going to be a job too far!

[Gawdzilla Sama]

Agh. Looks like you've got yourself something that was started in the days before databases. Short of using a spider-search as suggested above, I'm not sure there's a quick solution.

For a decent custom search you'd need to somehow import the data from each page as a record into a db and use that to serve pages dynamically. But I suspect that's going to be a job too far!

They're all the same, everything outside the table can be ditched.

So, basically, I want to import everything between "<P><TABLE" and "</TABLE></P>" into a file. I would want to do this to every html file in a directory.

[Tyrannical]

http://www.google.com/enterprise/search ... s_gss.html

Tadaa!

[Gawdzilla Sama]

That's nice, but this is just the first step. The data would be imported into a database after this step is done.

There is no overall index for the Naval History and Heritage Command's Library of Online Images, something I want to fix.

[Red Celt]

That's nice, but this is just the first step. The data would be imported into a database after this step is done.

There is no overall index for the Naval History and Heritage Command's Library of Online Images, something I want to fix.

Can be done with code (stripping out the table contents and writing that to a database), which could be used in web pages accessing the database. I'd offer my help, but I'm rustier than Davey Jones' cock ring.

[klr]

That's nice, but this is just the first step. The data would be imported into a database after this step is done.

There is no overall index for the Naval History and Heritage Command's Library of Online Images, something I want to fix.

I do databases - and importing data into and out of databases - for breakfast, dinner and tea. Quite literally - that's my job.

"We need to to talk ..."

[Gawdzilla Sama]

That's nice, but this is just the first step. The data would be imported into a database after this step is done.

There is no overall index for the Naval History and Heritage Command's Library of Online Images, something I want to fix.

I do databases - and importing data into and out of databases - for breakfast, dinner and tea. Quite literally - that's my job.

"We need to to talk ..."

The files are on Hyperwar. http://www.ibiblio.org/hyperwar/OnlineL ... rg11-2.htm

[klr]

Yup, I've seen 'em. I take it you'll want to retain a link to the picture along with each piece of text.

There are a number of sections making up the text that goes with each picture, so each of those might need to go into a separate field. Of course, there's probably no guarantee that the text for each and every picture is broken down the same way.

How many pictures are there in all do you think?

[Gawdzilla Sama]

Yup, I've seen 'em. I take it you'll want to retain a link to the picture along with each piece of text.

There are a number of sections making up the text that goes with each picture, so each of those might need to go into a separate field. Of course, there's probably no guarantee that the text for each and every picture is broken down the same way.

How many pictures are there in all do you think?

Official U.S. Navy Photograph, now in the collections of the National Archives.

Online Image: 91KB; 740 x 605 pixels

Reproductions of this image may also be available through the National Archives photographic reproduction system.

We would need everything above the first line, and only the second line of those three would be retained. Nothing below that would be needed.

I'm trying to figure out which drive the mirror is on here so I can get a rough file count. There is one HTML, one image and one thumbnail for each picture.

[Gawdzilla Sama]

Pictures.JPG (39.04 KiB) Viewed 4310 times

[klr]

I don't have FTP access yet, so I'm not sure what I can do at this point short of trying to download the entire site - probably not recommended ...

Some images also have a hi-res version:

http://www.ibiblio.org/hyperwar/OnlineL ... inct14.htm (third image down)

88,000+ photos. Since some of the HTML files have multiple images - see above - the number of HTML files will presumably be lower than that, even with two versions of some images.

[Gawdzilla Sama]

I don't have FTP access yet, so I'm not sure what I can do at this point short of trying to download the entire site - probably not recommended ...

Some images also have a hi-res version:

http://www.ibiblio.org/hyperwar/OnlineL ... inct14.htm (third image down)

88,000+ photos. Since some of the HTML files have multiple images - see above - the number of HTML files will presumably be lower than that, even with two versions of some images.

Did you get your account? I was told it was in process.

As for the files, I think it would be safe to divide by three (after deducting the number of folders) and taking 2-3% for hi-res copies, max. We can keep the same format, just a link to the hi-res.

[klr]

Ah, I see some emails from a few days ago that I'd missed. The details are there. I might need new software for this though, so it'll likely be tomorrow morning* when I get to it.

*Our politicians here in Ireland are going to be working very late tonight to debate important legislation, but I've no intention of staying up so late.