OP finds out that Drupal's default robots.txt doesn't actually cover his multi-language setup, which they configured to use path prefixes. An interesting question indeed. But is this really a Drupal-specific problem?

Very much so. In my opinion, there's no scenario where this question is off-topic here. It's not "his" multi-language setup, it's the multi-language setup, that Drupal core provides. Questions around that are welcome here.

And mutli-language is an optional feature of Drupal. Nothing the default robots.txt needs to cover from the start.

That's fine, and I can agree with your opinion on that. My opinion ectends further though, and is that as soon as language features are turned on, a Drupal-specific mechanism would kick in to solve the problem.

The same you wouldn't consider Drupal's default theme Bartik to cover all possible visual needs, you can't consider Drupal's default robots.txt to cover all possible path configurations.

No, but we're not considering whether Drupal's robots.txt covers "all possible path configurations". We're considering whether it covers the path configurations for the multi-language system that Drupal specifically provides out of the box.

If Bartik didn't work with the core multi-language setup, that would be a very Drupal-specific problem wouldn't it? In the same way, so is this.

Talking about robots.txt in specific there's http://www.robotstxt.org/robotstxt.html (and as SE network probably Webmasters) to get help configuring this generic file

Generic help isn't being sought, though - Drupal-specific help is being sought. That's why we have this site. I would imagine that's why you yourself answered the question, providing a Drupal-specific way for the OP to configure the very same "generic file".

robots.txt is SEO

Well no, not really. Some of its usage has some knock-on effect on SEO, sure, but that isn't what it's for. Its purpose is to instruct crawlers how to behave. Notice that the terms "SEO" and "Search engine optimisation" do not appear on http://www.robotstxt.org.

The only argument I can see for it being off topic in any way is that maybe it would be better served as a feature request to the core project. But as there are ways to solve the problem, I think that's reaching.

In my opinion, you're probably trying to apply the letter of the law too strictly here, without giving thought to the nuances of the question/subject, and how applicable and useful this is and will be to future users of Drupal.

OP finds out that Drupal's default robots.txt doesn't actually cover his multi-language setup, which they configured to use path prefixes. An interesting question indeed. But is this really a Drupal-specific problem?

Very much so. In my opinion, there's no scenario where this question is off-topic here. It's not "his" multi-language setup, it's the multi-language setup, that Drupal core provides. Questions around that are welcome here.

And multi-language is an optional feature of Drupal. Nothing the default robots.txt needs to cover from the start.

That's fine, and I can agree with your opinion on that. My opinion extends further though, and is that as soon as language features are turned on, a Drupal-specific mechanism would kick in to solve the problem.

The same you wouldn't consider Drupal's default theme Bartik to cover all possible visual needs, you can't consider Drupal's default robots.txt to cover all possible path configurations.

No, but we're not considering whether Drupal's robots.txt covers "all possible path configurations". We're considering whether it covers the path configurations for the multi-language system that Drupal specifically provides out of the box.

If Bartik didn't work with the core multi-language setup, that would be a very Drupal-specific problem wouldn't it? In the same way, so is this.

Talking about robots.txt in specific there's http://www.robotstxt.org/robotstxt.html (and as SE network probably Webmasters) to get help configuring this generic file

Generic help isn't being sought, though - Drupal-specific help is being sought. That's why we have this site. I would imagine that's why you yourself answered the question, providing a Drupal-specific way for the OP to configure the very same "generic file".

robots.txt is SEO

Well no, not really. Some of its usage has some knock-on effect on SEO, sure, but that isn't what it's for. Its purpose is to instruct crawlers how to behave. Notice that the terms "SEO" and "Search engine optimisation" do not appear on http://www.robotstxt.org.

The only argument I can see for it being off topic in any way is that maybe it would be better served as a feature request to the core project. But as there are ways to solve the problem, I think that's reaching.

In my opinion, you're probably trying to apply the letter of the law too strictly here, without giving thought to the nuances of the question/subject, and how applicable and useful this is and will be to future users of Drupal.

OP finds out that Drupal's default robots.txt doesn't actually cover his multi-language setup, which they configured to use path prefixes. An interesting question indeed. But is this really a Drupal-specific problem?

Very much so. In my opinion, there's no scenario where this question is off-topic here. It's not "his" multi-language setup, it's the multi-language setup, that Drupal core provides. Questions around that are welcome here.

The same you wouldn't consider Drupal's default theme Bartik to cover all possible visual needs, you can't consider Drupal's default robots.txt to cover all possible path configurations.

No, but we're not considering whether Drupal's robots.txt covers "all possible path configurations". We're considering whether it covers the path configurations for the multi-language system that Drupal specifically provides out of the box.

If Bartik didn't work with the core multi-language setup, that would be a very Drupal-specific problem wouldn't it? In the same way, so is this.

Talking about robots.txt in specific there's http://www.robotstxt.org/robotstxt.html (and as SE network probably Webmasters) to get help configuring this generic file

Generic help isn't being sought, though - Drupal-specific help is being sought. That's why we have this site. I would imagine that's why you yourself answered the question, providing a Drupal-specific way for the OP to configure the very same "generic file".

robots.txt is SEO

Well no, not really. Some of its usage has some knock-on effect on SEO, sure, but that isn't what it's for. It's purpose is to instruct crawlers how to behave. Notice that the terms "SEO" and "Search engine optimisation" do not appear on http://www.robotstxt.org.

The only argument I can see for it being off topic in any way is that maybe it would be better served as a feature request to the core project. But as there are ways to solve the problem, I think that's reaching.

In my opinion, you're probably trying to apply the letter of the law too strictly here, without giving thought to the nuances of the question/subject, and how applicable and useful this is and will be to future users of Drupal.

OP finds out that Drupal's default robots.txt doesn't actually cover his multi-language setup, which they configured to use path prefixes. An interesting question indeed. But is this really a Drupal-specific problem?

Very much so. In my opinion, there's no scenario where this question is off-topic here. It's not "his" multi-language setup, it's the multi-language setup, that Drupal core provides. Questions around that are welcome here.

And mutli-language is an optional feature of Drupal. Nothing the default robots.txt needs to cover from the start.

That's fine, and I can agree with your opinion on that. My opinion ectends further though, and is that as soon as language features are turned on, a Drupal-specific mechanism would kick in to solve the problem.

The same you wouldn't consider Drupal's default theme Bartik to cover all possible visual needs, you can't consider Drupal's default robots.txt to cover all possible path configurations.

No, but we're not considering whether Drupal's robots.txt covers "all possible path configurations". We're considering whether it covers the path configurations for the multi-language system that Drupal specifically provides out of the box.

If Bartik didn't work with the core multi-language setup, that would be a very Drupal-specific problem wouldn't it? In the same way, so is this.

Talking about robots.txt in specific there's http://www.robotstxt.org/robotstxt.html (and as SE network probably Webmasters) to get help configuring this generic file

Generic help isn't being sought, though - Drupal-specific help is being sought. That's why we have this site. I would imagine that's why you yourself answered the question, providing a Drupal-specific way for the OP to configure the very same "generic file".

robots.txt is SEO

Well no, not really. Some of its usage has some knock-on effect on SEO, sure, but that isn't what it's for. Its purpose is to instruct crawlers how to behave. Notice that the terms "SEO" and "Search engine optimisation" do not appear on http://www.robotstxt.org.

The only argument I can see for it being off topic in any way is that maybe it would be better served as a feature request to the core project. But as there are ways to solve the problem, I think that's reaching.

In my opinion, you're probably trying to apply the letter of the law too strictly here, without giving thought to the nuances of the question/subject, and how applicable and useful this is and will be to future users of Drupal.