[SATLUG] Security Issues

Jonathan Kelley jonkelley at gmail.com
Thu Jun 26 10:09:35 CDT 2014


Christopher,

I'm assuming you write a reactor pattern or something for spidering in
python? I'd think the barrier to threading etc would make it harder to code
a spider in python.

--
Jon

* JonK gives kris a colourful gift box with question marks and tux's
printed all over. "Open it!"
* Kris looks around nervously and slowly opens it
<JonK> it's empty!
<JonK> the box represents what an open-source linux programmer makes in a
year


On Thu, Jun 26, 2014 at 5:43 AM, Daniel Villarreal <youcanlinux at gmail.com>
wrote:

> Can one really count on robots.txt being adhered to?
> Daniel
>
> On Wed, Jun 25, 2014 at 4:43 PM, Christopher Lemire
> <christopher.lemire at gmail.com> wrote:
> > As far as what search engines include, see also what's in the
> > robots.txt of the root directory of apache's home directory. And then
> > secure what's hidden from web crawlers/spiders there. Google doesn't
> > index those in it's search results, but web server admins SHOULD know
> > that simply hiding what's there be used to stop attacks.
> >
> > Christopher Lemire
> > --
> > _______________________________________________
> > SATLUG mailing list
> > SATLUG at satlug.org
> > http://alamo.satlug.org/mailman/listinfo/satlug to manage/unsubscribe
> > Powered by Rackspace (www.rackspace.com)
> --
> _______________________________________________
> SATLUG mailing list
> SATLUG at satlug.org
> http://alamo.satlug.org/mailman/listinfo/satlug to manage/unsubscribe
> Powered by Rackspace (www.rackspace.com)
>


More information about the SATLUG mailing list