Hello and welcome to our community! Is this your first visit?
Register
Enjoy an ad free experience by logging in. Not a member yet? Register.
Results 1 to 5 of 5
  1. #1
    Senior Coder
    Join Date
    Dec 2005
    Location
    Slovenia
    Posts
    1,994
    Thanks
    120
    Thanked 76 Times in 76 Posts

    crawler - pointing it where to go

    for some reason all boots are repeatedly looking here:
    http://agrozoo.net/jsp/Tools.jsp
    but rarely to other of my pages on that site

    So I included this into 'Tools.jsp' page:
    Code:
    <a  style ="visibility:hidden" href="../jsp/Galery_list_of_specieses.jsp">dfgsdfg</a>
    which points to:
    http://agrozoo.net/jsp/Galery_list_of_specieses.jsp

    Hope this will make crawlers pick that list and links. m'I right ?
    Found a flower or bug and don't know what it is ?
    agrozoo.net galery
    if you don't spot search button at once, there is search form:
    agrozoo.net galery search

  • #2
    Senior Coder
    Join Date
    Dec 2005
    Location
    Slovenia
    Posts
    1,994
    Thanks
    120
    Thanked 76 Times in 76 Posts
    in another words:

    How are things done on coding forums , that in a time period of one hour after submiting this post, this post becomes third hit in google, typing in "agrozoo.net" ?
    Last edited by BubikolRamios; 02-13-2010 at 12:00 PM.
    Found a flower or bug and don't know what it is ?
    agrozoo.net galery
    if you don't spot search button at once, there is search form:
    agrozoo.net galery search

  • #3
    Senior Coder
    Join Date
    Jul 2005
    Location
    UK
    Posts
    1,051
    Thanks
    6
    Thanked 13 Times in 13 Posts
    You have a nofollow directive in the page source:

    Code:
    <meta name="robots" content="nofollow" />
    This prevents search engines from following any of the links (including your new hidden one), which is why nothing else gets crawled.

    You should remove that hidden link, by the way - it's against search engine guidelines.

    Codingforums gets crawled and indexed quickly because it's a very popular site that lots of people link to on a regular basis - and because it doesn't block crawlers from accessing pages with rogue meta tags!

  • #4
    Senior Coder
    Join Date
    Dec 2005
    Location
    Slovenia
    Posts
    1,994
    Thanks
    120
    Thanked 76 Times in 76 Posts
    jeah I have this nofollow and boot stops in robots txt for a reason. Forinstance:
    http://www.google.si/search?hl=sl&rl...meta=&aq=f&oq=

    What does this find ? A page where "phaonia+subventa" was once, and crawler picked it up there at the time, it is still on site but not there, pushed forward.

    hence I created
    http://agrozoo.net/jsp/Galery_list_of_specieses.jsp
    which has no robots stops and presents things to crawlers in clear way.

    using this: http://www.auditmypc.com/free-sitemap-generator.asp
    to check if google will see the pages I want, and it does according to that tool. Submited pagemap.xml to google a couple of days ago, no hits to ".../jsp/Galery_list_of_specieses" jet ....
    Last edited by BubikolRamios; 02-16-2010 at 03:36 PM.
    Found a flower or bug and don't know what it is ?
    agrozoo.net galery
    if you don't spot search button at once, there is search form:
    agrozoo.net galery search

  • #5
    Senior Coder
    Join Date
    Dec 2005
    Location
    Slovenia
    Posts
    1,994
    Thanks
    120
    Thanked 76 Times in 76 Posts
    just checked what google tool for web masters did with my sitemap.xml , and it says:

    670 submited links
    23 indexed links

    Doest this menan that this is it, or will it process the rest allso in time ?
    Found a flower or bug and don't know what it is ?
    agrozoo.net galery
    if you don't spot search button at once, there is search form:
    agrozoo.net galery search


  •  

    Posting Permissions

    • You may not post new threads
    • You may not post replies
    • You may not post attachments
    • You may not edit your posts
    •