Thank you for your help
When accessing a web page, read images, CSS, JS, etc. from another server.
I try to cash it.
I use Apache's mod_cache cache.
When access increases, the following error occurs, followed by
I'm having trouble with the problem of not being able to cash at all
[Mon Jan 05 09:43:37 2015] [error] (103) Software caused connection abort:cache: error returned while trying to return disk cached data
The conf configuration file is as follows
##################################################################
#
# Location Area
#
##############################
<Location"/qr_cu/">
ProxyPass balancer://panda_customer_cluster/
ProxyPassReverse balancer://panda_customer_cluster/
</Location>
<Location"/qr_cl/">
ProxyPass balancer://panda_client_cluster/
ProxyPassReverse balancer://panda_client_cluster/
</Location>
<Location"/portal_cu/">
ProxyPass balancer://koala_customer_cluster/
ProxyPassReverse balancer://koala_customer_cluster/
</Location>
<Location"/idv_tl/">
ProxyPass balancer://tool_cluster/
ProxyPassReverse balancer://tool_cluster/
</Location>
<Location"/tgb_cu/">
ProxyPass balancer://owl_customer_cluster/
ProxyPassReverse balancer://owl_customer_cluster/
</Location>
<Location"/tgb_cl/">
ProxyPass balancer://owl_client_cluster/
ProxyPassReverse balancer://owl_client_cluster/
</Location>
##############################
#
# Proxy balancer Area
#
##############################
<Proxy balancer://panda_customer_cluster>
SetOutputFilter DEFLATE
BrowserMatch^Mozilla/4gzip-only-text/html
BrowserMatch^Mozilla/4\.0 [678] no-gzip
BrowserMatch\bMSI[E]!no-gzip!gzip-only-text/html
SetEnvIfNoCase Request_URI\.(?:gif|jpe?g|png|ico)$no-gzip dont-vary
Header append Vary User-Agent env=!dont-vary
BalancerMember http://xxx.xxx.xxx.xxx
</Proxy>
<Proxy balancer://panda_client_cluster>
BalancerMember http://xxx.xxx.xxx.xxx
</Proxy>
<Proxy balancer://koala_customer_cluster>
SetOutputFilter DEFLATE
BrowserMatch^Mozilla/4gzip-only-text/html
BrowserMatch^Mozilla/4\.0 [678] no-gzip
BrowserMatch\bMSI[E]!no-gzip!gzip-only-text/html
SetEnvIfNoCase Request_URI\.(?:gif|jpe?g|png|ico)$no-gzip dont-vary
Header append Vary User-Agent env=!dont-vary
BalancerMember http://xxx.xxx.xxx.xxx
</Proxy>
<Proxy balancer://tool_cluster>
BalancerMember http://xxx.xxx.xxx.xxx
</Proxy>
<Proxy balancer://owl_customer_cluster>
BalancerMember http://xxx.xxx.xxx.xxx
</Proxy>
<Proxy balancer://owl_client_cluster>
BalancerMember http://xxx.xxx.xxx.xxx
</Proxy>
##############################
#
# Chache Area
#
##############################
<IfModule mod_cache.c>
<IfModule mod_disk_cache.c>
CacheIgnoreCacheControl On
CacheIgnoreNoLastMod On
CacheIgnoreHeaders Set-Cookie User-Agent Vary Cookie Server
CacheDefaultExpire86400
CacheRoot/var/www/xxxxxx/image/cache
CacheEnable disk /qr_cu/
CacheEnable disk /qr_cl/
CacheEnable disk/portal_cu/
CacheEnable disk /idv_tl/
CacheEnable disk /tgb_cu/
CacheEnable disk /tgb_cl/
CacheDirLevels 6
CacheDirLength3
CacheMaxFileSize 10000000
</IfModule>
</IfModule>
I've been looking around here (Apache/mod_cache: "Error 103 Software caused connection abort"), but I haven't found a clear answer yet. Do you have any idea?
Thank you for your cooperation.
Additional information
Thank you for your reply.
I tried to increase the number of connections, but it didn't change much.
However, when I checked the free memory, I found out that there was only about 120MB left on average, and I guess they ate it all up when access increased. What do you think?
-bash-4.1$free
total used free shared buffers cached
Mem: 3924856 3803180 1216760 1529244 1146440
-/+ buffers/cache: 1127496 2797360
Swap: 2097148 13844 2083304
Additional information.There was a mod_proxy error, so I will add it too
The following error occurs when increasing access:
[Mon Jan 05 12:06:56 2015] [error] (70007) The timeout specified has expired: proxy: HTTP: attempt to connect to xxx.xxx.xxx.xxx:80 (xxx.xxx.xxx.xxx) failed
[Mon Jan 05 12:06:56 2015] [error] ap_proxy_connect_backend disable worker for (xxx.xxx.xxx.xxx)
[Mon Jan 05 12:08:29 2015] [error] (70007) The timeout specified has expired: proxy: HTTP: attempt to connect to xxx.xxx.xxx.xxx:80 (xxx.xxx.xxx.xxx) failed
[Mon Jan 05 12:08:29 2015] [error] ap_proxy_connect_backend disable worker for (xxx.xxx.xxx.xxx)
[Mon Jan 05 12:14:19 2015] [error] (70007) The timeout specified has expired: proxy: HTTP: attempt to connect to xxx.xxx.xxx.xxx:80 (xxx.xxx.xxx.xxx) failed
[Mon Jan 05 12:14:19 2015] [error] ap_proxy_connect_backend disable worker for (xxx.xxx.xxx.xxx)
It seems to be a mod_proxy error...
Thank you for your cooperation.
Below is a part of the mod_cache source code (mod_cache.c), where the example error message appears.
/*kick off the filter stack*/
out = apr_brigade_create(r->pool,r->connection->bucket_alloc);
rv = ap_pass_brigade(r->output_filters, out);
if(rv!=APR_SUCCESS){
if(rv!=AP_FILTER_ERROR){
ap_log_error(APLOG_MARK, APLOG_ERR, rv, r->server,
"cache:error returned while trying to return%s"
"cached data",
cache->provider_name);
}
return rv;
}
The reason for the problem is that ap_pass_brigade returns false, but the reason why ap_brigate_create before it seems to have a problem.I can't say for sure because I don't have a real environment at hand, but I'm guessing that the connection pool is tight because access to the Apache server process or thread is concentrated.
I remember that there was a variable in Apache setting that set the maximum number of connections (I'm sorry if I'm wrong), but why don't you increase it?
I'm not confident at all because the information is a little old, but there seems to be an example where cacheDirLength is reduced by skipping the cache from 2 to 1.Is it due to the decrease in the number of subdirectories(Maybe it works for ext2/3)
Also, the SO information in the question was Apache/2.2.14.
I looked at Changes, but I can read mod_cache as if it were being modified frequently
Recently, there are traces of fighting against Thundering Herd issue in Vary header. (It seems to have nothing to do with this case because it was used with CacheLock.)
I'm not confident about this either, but the latest 2.2 series or 2.4.x might solve the problem.
(Also, I think I'm going to ruin my hard work by caching, but I'm going to adjust the Vary header and try to change the disk I/O...)
Red Hat has information about mod_cache error messages.
Error, apache, 103 Software caused connection abort (Subscription agreement is required to view)
To sum up,
That's right.
In addition to the mod_proxy error message, the browser timed out while waiting for the connection to Backend, and the connection between the browser and httpd was interrupted, resulting in the mod_cache error message.If the httpd timeout is shorter than the browser timeout value and the server times out first, mod_cache error messages may be suppressed.
As for mod_proxy, the connection to Backend is timed out, so I think the processing on the backend side is not in time. The mod_proxy side has more SYN_SENT in the output of "netstat-tan" (assuming Linux) at the time of the failure. You may want to check each load or see if there are any error messages (such as /var/log/messages) such as insufficient OS resources.
As for memory usage, buffers (cache of block device cache.I don't think this is a problem in itself, but if there is a lot of io_wait (in vmstat's wa column), it will cause the load on the httpd side (load average) to go up because it is waiting to write to the disk.
© 2024 OneMinuteCode. All rights reserved.