Google doesn't render Ember app


#1

Hi all! I have tested my app by “Fetch as Google” and got empty page: https://gyazo.com/2b28487ac1a25e11e2e87888779e3f2a.

Evidently google crawler thinks that there is just background without data. Of course, in any browser app looks completely different: https://gyazo.com/009a5a9719f80aef70fc22bc3d777cba.

So is it default behavior for Ember app and it just can’t be indexed by gooogle without server renderering? Or I just did something wrong in my code?

Here is code btw: https://github.com/drinkit/ember-client


#2

Generally google renders your app for SEO purposes. I don’t know what tool your were using to check this, but google definitely indexes some ember apps just fine.

If you’re not showing up in the index, or you’re looking to render static html for other websites, look into fastboot: http://www.ember-fastboot.com/


#3

I used official google tool for checking how crawler is acting: https://support.google.com/webmasters/answer/6066468?hl=en. I want to believe that google really indexes some ember apps, but it definetly doesn’t want to index mine for some reasons.


#4

Google nicely run and understand native Ember apps, without fastboot, however Fastboot an extra and improve user experience, improved user experience leads to better SEO.

You probably would like to check:

  • your robots.txt
User-agent: *
Allow: /
  • sitemap built and added to Google Search Console. (I don’t see any link in your sitemap, should point to the articles/products.)

  • Maybe worth to implement a product/article title slug route instead of id, so the url would be more user friendly.


#5

I have next robots.txt:

# http://www.robotstxt.org
User-agent: *
Disallow:

Sitemap: http://drinkit.guru/sitemap.xml

As I understand it means “allow all for all robots”. I will try your version of robots.txt. Maybe its some difference here for google crawler.

My main goal now to achieve at least something with google. Sitemap and pretty urls are great but optional at this moment.

What else could cause this problem? Maybe some ember configuration… or just my bad code?)


#6

(Yep, I saw your robots.txt and your sitemap before… you right it shouldn’t block robots, however worth checking with Google Search Console.)

So, verify your page here https://www.google.com/webmasters/tools/

And look around, check all the options, use the tool to send your page for crawlers, check how google see your page, etc…

Happy growth hacking! :smiley:


#7

I already verified my page (screenshots from google tools) and google see nothing. Do you have examples of some ember apps that indexed and rendered by google without problems? It will be really helpful for me now =).


#8

Google do index Ember apps just fine? This is new to me…

As far as i know, Google do not render javascript content… So, Ember apps should not get indexed at all -, at best, the meta-tags and some basic data inserted directly in the index.htm file…

I never used Fast boot before, but based on a quick read, it renders Ember apps in backend with node, and shows it as a static page, right? In this case, it would work just fine…


#9

I just checked my very basic demo app, which is hosted on github pages, simple stuff, but when I search for text which is exist on the page, it appears in google result list: https://www.google.co.nz/search?q=bookstore2+bookstore-client


#10

Google understand and run javascript content when indexing your webpage. http://searchengineland.com/tested-googlebot-crawls-javascript-heres-learned-220157, https://webmasters.googleblog.com/2014/05/understanding-web-pages-better.html


#11

Yeah, I also test it and google found, for example “zoltan bookstore The Killer Doctors” at your site. I am encouraged by this but I am confused also why google tools can’t render my page… Could you check how your site will be rendered by google crawler https://support.google.com/webmasters/answer/6066468?hl=en ? Maybe it’s some problem in google tool :slight_smile:


#12

There is an other demo app, what I published on github pages. http://zoltan.nz/contacts-app-client/ and I realized, I already used that tool, what you just linked.

This is looking good also.


#13

Awesome! Thank you very much! Now, I just need to recognize what is wrong with my code =)


#14

Hey Yaroslav, this is your website? http://drinkit.guru As I see it is an Adobe Flash page, totally invisible for google.

Or you deployed your ember page somewhere else?


#15

http://ember.drinkit.guru. At main domain deployed old deprecated version. New awesome ember version is still in development. I tried to test it crawl-ability before deploying to prod.


#16

When I checked your old home page, it looked, that there is a weird redirect… probably that flash framework does something, however, google could recognize that as cloaking, so it could lead for banning from the search engine. I would suggest, that deploy a static homepage or the new Ember app as soon as possible.


#17

I finally deployed my app to prod - https://drinkit.guru, but it still doesn’t want to be rendered by google tool. I have no clue what I am doing wrong… I checked robots.txt many time and sitemap.xml as well.

I tried to change something in my .htaccess but it didn’t help… Here it is btw:

# Set the cache-control max-age
# 1 year
<FilesMatch ".(ico|pdf|flv|jpg|jpeg|png|gif|js|css|swf|ttf|svg)$">
Header set Cache-Control "max-age=31449600, public"
</FilesMatch>
# 2 DAYS
<FilesMatch ".(xml|txt)$">
Header set Cache-Control "max-age=172800, public, must-revalidate"
</FilesMatch>
# 4 HOURS
<FilesMatch ".(html|htm)$">
Header set Cache-Control "max-age=14400, must-revalidate"
</FilesMatch>
# Turn off the ETags
Header unset ETag
FileETag None

# Turn off the Last Modified header except for html docs
<FilesMatch ".(ico|pdf|flv|jpg|jpeg|png|gif|js|css|svg)$">
Header unset Last-Modified
</FilesMatch>

<IfModule mod_rewrite.c>
RewriteEngine On
RewriteRule ^sitemap.xml$ https://prod-drunkedguru.rhcloud.com/rest/sitemap [R=301,L]
RewriteRule ^index\.html$ - [L]
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule (.*) index.html [L]
RewriteCond %{HTTPS} off
RewriteRule .* https://%{HTTP_HOST}%{REQUEST_URI} [L,R=301]
</IfModule>

I will be very appreciate for any hint!

UPDATE:

I have deployed old simper version of my app, and its rendered. The next commit (where I added some new stuff to site) doesn’t work with google. But when I have removed this new stuff from latest app version it doesn’t want to be rendered too. Now I have guess that it related to page load or page size.


#18

My guess is there is a time to wait in the crawler, and you’re API requests are taking longer than that wait time to resolve.


#19

After a day of investigation I found two problems those prevented pages from crawling.

1.Validation component:

import Ember from 'ember';

const {
  computed,
  observer,
  defineProperty,
  run
} = Ember;

export default Ember.Component.extend({
  classNames: ['form-group', 'has-feedback', 'validated-input'],
  classNameBindings: ['isValid:has-success', 'showErrorClass:has-error'],
  isValid: false,
  model: null,
  value: null,
  rawInputValue: null,
  type: 'text',
  valuePath: '',
  placeholder: '',
  attributeValidation: null,
  isTyping: false,

  didValidate: computed.oneWay('targetObject.didValidate'),

  showErrorClass: computed('isTyping', 'showMessage', 'hasContent', 'attributeValidation', function() {
    return this.get('attributeValidation') && !this.get('isTyping') && this.get('showMessage');
  }),

  hasContent: computed.notEmpty('rawInputValue'),

  isValid: computed.and('hasContent', 'attributeValidation.isValid'),

  isInvalid: computed.oneWay('attributeValidation.isInvalid'),

  inputValueChange: observer('rawInputValue', function() {
    this.set('isTyping', true);
    run.debounce(this, this.setValue, 500, false);
  }),

  showMessage: computed('attributeValidation.isDirty', 'isInvalid', 'didValidate', function() {
    return (this.get('attributeValidation.isDirty') || this.get('didValidate')) && this.get('isInvalid');
  }),

  setValue() {
    this.set('value', this.get('rawInputValue'));
    this.set('isTyping', false);
  },

  init() {
    this._super(...arguments);
    var valuePath = this.get('valuePath');
    defineProperty(this, 'attributeValidation', computed.oneWay(`model.validations.attrs.${valuePath}`));
    this.set('rawInputValue', this.get(`model.${valuePath}`));
    defineProperty(this, 'value', computed.alias(`model.${valuePath}`));
  }
});

2.This piece of code (I wrote it for performance optimization):

  ingredients: function() {
    var self = this;
    return DS.PromiseArray.create({
      promise: new Promise((resolve, reject) => {
        let loadedIngredients = self.store.peekAll('ingredient');
        if (loadedIngredients && loadedIngredients.get('length') > 0) {
          let mappedIngredients = self.get('recipe').get('ingredientsWithQuantities').map(function(item) {
            return {
              name: self.store.peekRecord('ingredient', item.ingredientId).get('name'),
              quantity: item.quantity
            };
          });
          resolve(mappedIngredients);
        } else {
          this.store.findAll('ingredient').then(function() {
            let mappedIngredients = self.get('recipe').get('ingredientsWithQuantities').map(function(item) {
              return {
                name: self.store.peekRecord('ingredient', item.ingredientId).get('name'),
                quantity: item.quantity
              };
            });
            resolve(mappedIngredients);
          })
        }
      })
    });
  }.property('recipe.ingredientsWithQuantities')

After removing/changing I finally got something similar to success:

I hope it will be enough for google to properly renders and indexes my pages,