|
Badger Is Fourth-From-The-Top^
| "The difference between genius and stupidity is that genius has its limits." - Albert Einstein | | "As far as we know, our computer has never had an undetected error." - Weisert | | "If you are searching for perfection in others, then you seek disappointment. If you are seek perfection in yourself, then you will find failure." - Balboos HaGadol Mar 2010 |
|
|
|
|
|
Hello,
I would like to build up a large collection of sample files for testing purposes and nothing is better than real data that is out there. So what I was thinking is a program to go out on the internet like a webcrawler but just look for files that are linked for download on the internet and download these into an output folder. Would like files of any type but okay aso if specifying types like pdf, doc, xls, docx, ppt, txt, etc.
Does something like this already exist? I would not like to recreate the wheel if someone already has something that does this.
Thank you,
Larry
|
|
|
|
|
You mean, you want a virus sucker ?
~RaGE();
I think words like 'destiny' are a way of trying to find order where none exists. - Christian Graus
Do not feed the troll ! - Common proverb
|
|
|
|
|
Nothing related to viruses please.
|
|
|
|
|
Larry Wall wrote: just look for files that are linked for download on the internet and download these into an output folder
Um.
I suspect that a very large number of the files you "find" will not be Safe For Work... 
The only instant messaging I do involves my middle finger.
English doesn't borrow from other languages.
English follows other languages down dark alleys, knocks them over and goes through their pockets for loose grammar.
|
|
|
|
|
The way I'd go about this would be to start from a google search, which allows you to find files based on type. Then crawl that result set normally.
|
|
|
|
|
I used to do this all the time, but I think more recently google has started properly NOT returning files in some circumstances (whether that is based on extension or server path, or both I cannot remember). By that I mean, I no longer find files with particular extensions like I used to.
As an aside, I used to work in a very poorly documented and obtuse software language. The only way to figure out how to use it was search the web for files with the same extension and that contained the method I wanted to use... that was my mode of documentation  Oddly, everyone I worked with thought it was amazing the things I could do with that system  even though they had 10 - 20 years experience on me.
|
|
|
|
|
You could use my Domain Walker[^] utility and filter on file types of interest. Domain Walker utilizes this[^] framework to access content from the web.
/ravi
PS: You have a very famous[^] name. I was going to suggest you just use Perl. 
|
|
|
|
|
Hello Ravi,
Yes I know. I get asked probably about once a month if I designed Perl. I will check Domain Walker and see if I can use it for this purpose. Thank you for this.
Good day,
Larry
|
|
|
|
|
|
Thank you Madspudster, I will check this out soon and see if this will work for me. Good day, Larry
|
|
|
|
|
"Madspudster" ... that's a new one!! 
|
|
|
|
|
Hi,
Just wasting a little time (too late in the day to start anything new) when I looked at my profile and found my Bob looking like an extra from Tarantino movie.
|
|
|
|
|
When you added the extra info that changes you from "Member" to "Professional".
Think yourself lucky: in one of mine he is wearing a toga, and the other...well...nude.
Fortunately, the "MVP" picture is self censored...
The only instant messaging I do involves my middle finger.
English doesn't borrow from other languages.
English follows other languages down dark alleys, knocks them over and goes through their pockets for loose grammar.
|
|
|
|
|
OriginalGriff wrote: self censored
Actually not, the shield is behind Bob.
~RaGE();
I think words like 'destiny' are a way of trying to find order where none exists. - Christian Graus
Do not feed the troll ! - Common proverb
|
|
|
|
|
That's "Protector" rather than "MVP": http://www.codeproject.com/script/Membership/Types.aspx[^]
The only instant messaging I do involves my middle finger.
English doesn't borrow from other languages.
English follows other languages down dark alleys, knocks them over and goes through their pockets for loose grammar.
|
|
|
|
|
|
hi MehGerbil,
i am so tired because i have just received an email from Telerik with the subject title [Invitation] Kendo UI Release Keynote: Responsive UI for Modern Apps
hint: sender email address is webinars at kendoui dot com
|
|
|
|
|
affsdfafdafsdafsdfasdaeffae afe a fa as s asd sd
asd
af f
as
d fa
sd fasdf asfasdafsdfdafasd
^ Sorry, reading that caused me to pass out.
|
|
|
|
|
An extension to your contacts or text message client that allows you to mark someone as your partner and then automatically strips out all absentmindedly placed x from text messages to all other contacts.
That would be useful thanks.
“I believe that there is an equality to all humanity. We all suck.” Bill Hicks
|
|
|
|
|
|
That's a great idea!
xxx XD
MVVM # - I did it My Way
___________________________________________
Man, you're a god. - walterhevedeich 26/05/2011
.\\axxx
(That's an 'M')
|
|
|
|
|
I'm installing a new version of the JDK; doing work on one monitor while the installer spins away on the second. i happened to glance to the side and saw the banner "over 3 billion devices run Java". That made me wonder, are they counting Android phones in the total? Google says they run Java and 99% of developers would agree and it's naturally in Whoracle's interest to show as big a number as possible; but the Whoracle-Google lawsuit was in part about the former claiming that because Google forked it it wasn't really Java any more.
Did you ever see history portrayed as an old man with a wise brow and pulseless heart, waging all things in the balance of reason?
Is not rather the genius of history like an eternal, imploring maiden, full of fire, with a burning heart and flaming soul, humanly warm and humanly beautiful?
--Zachris Topelius
Training a telescope on one’s own belly button will only reveal lint. You like that? You go right on staring at it. I prefer looking at galaxies.
-- Sarah Hoyt
|
|
|
|
|
They were claiming that Java runs on "over 3 billion devices" years ago. Even before Android became popular. I'm still wondering which devices they mean 
Besides, just because Java is used for the runtime doesn't mean it runs applications created for the JRE.
|
|
|
|
|
There really are a tremendous number of consumer electronic devices that run JREs, embedded or otherwise, under the hood.
It's cool and creepy in about equal measure. I don't have any trouble believing that number.
|
|
|
|
General 
News 
Suggestion 
Question 
Bug 
Answer 
Joke 
Praise 
Rant 
Admin 
Submit an article or tip
