I want to handle Net::OpenTimeout (execution expired) with Net::HTTP

Asked 2 years ago, Updated 2 years ago, 425 views

Prerequisites

Currently, we are using Ruby on Rails to send a GET request to Google Books Api to process the response and return Json data to Nxt.js.

We have created a function to send a GET request to Google Books Api using the method defined in the following model in Rails.
It works fine in the local environment. When I deployed to aws(ecs, EC2) an error occurred as the title suggests.

I've been stuck for a few days, so I'm asking you a question to help me.

Additional
We ask the same question on this site.
If it is resolved, I will report it to both sites and share the solution to help those who have similar problems with me.
https://teratail.com/questions/dljdho1ts6d9ha

9From September 4th
The above site has already solved the problem because we understand that the Rails Ec2 container has not sent a get request to GoogleApi (the Ec2 container has not been communicated with GoogleApi).
However, the goal of "I want to send a GET request to Google Books Api via Net::HTTP for a response" has not been met, so I have included progress in the supplementary information until the investigation is continued and resolved.

What do you want to do

● I want to know what's going on.
● Net:: I want to deal with Net::OpenTimeout (execution expired) with HTTP.

Problems/Error Logs Occurring

● Logs the aws environment

Started GET"/api/v1/books/search/?keyword=%E3%83%86%E3%82%B9%E3%83%88" for 000.000.00.000 at 2022-09-03 16:11:58+0900
Processing by Api::V1::BooksController #search as HTML
Parameters: {"keyword"=>"........."}
https://www.googleapis.com/books/v1/volumes?q=............&country=JP
https://www.googleapis.com/books/v1/volumes?q=%E3%83%86%E3%82%B9%E3%83%88&country=JP
https://www.googleapis.com/books/v1/volumes?q=%E3%83%86%E3%82%B9%E3%83%88&country=JP
# <Net::HTTP:0x00007f7e4c5fd008>
# <Net::HTTP::Get:0x00007f7e4c5fcdb0>
Completed 500 Internal Server Error in 60002ms (ActiveRecord: 0.0ms | Allocations: 2006)
↓ Log data that may be relevant
Net::OpenTimeout(execution expired):
/usr/local/lib/ruby/3.0.0/net/http.rb:987:in `initialize'
/usr/local/lib/ruby/3.0.0/net/http.rb:987:in `open'
/usr/local/lib/ruby/3.0.0/net/http.rb:987:in `block in connect'
/usr/local/lib/ruby/3.0.0/timeout.rb:107:in `timeout'
/usr/local/lib/ruby/3.0.0/net/http.rb:985:in `connect'
/usr/local/lib/ruby/3.0.0/net/http.rb:970:in `do_start'
 /usr/local/lib/ruby/3.0.0/net/http.rb:959:in `start'
/usr/local/lib/ruby/3.0.0/net/http.rb:1512:in `request'
↑ Log data that may be relevant
/app/models/google_book.rb:61:in`search'

●Log local environment

Started GET"/api/v1/books/search/?keyword=%E3%83%86%E3%82%B9%E3%83%88" for 000.00.0.0 at 2022-09-03 16:03:24+0900
Processing by Api::V1::BooksController #search as HTML
  Parameters: {"keyword"=>"........."}
https://www.googleapis.com/books/v1/volumes?q=............&country=JP
https://www.googleapis.com/books/v1/volumes?q=%E3%83%86%E3%82%B9%E3%83%88&country=JP
https://www.googleapis.com/books/v1/volumes?q=%E3%83%86%E3%82%B9%E3%83%88&country=JP
# <Net::HTTP:0x00007fd7a0119e58>
# <Net::HTTP::Get:0x00007fd7a0119930>
# <Net::HTTPOK:0x00007fd7a0125550>
200
OK.
Completed 200 OK in 692ms (Views: 0.7ms | ActiveRecord: 0.0ms | Allocations: 4250)

Source Codes Affected

models/google_book.rb

require'google_books_api'

class GoogleBook
  include ActiveModel::Model
  include ActiveModel::Attributes
  include ActiveModel::Validations
  
  attribute:google_books_api_id, :string
  attribute:authors
  attribute:image,:string
  attribute:published_at, :date
  attribute:title,:string
  attribute:publisher,:string
  
  
  values:google_books_api_id, presence:true
  values:title, presence:true
  
  class<<self
  # Use modules created below lib
  include GoogleBooksApi
  
    def new_from_item(item)
      @item=item
      @volume_info=@item ['volumeInfo']
      new(
        google_books_api_id:@item['id'],
        authors: @volume_info ['authors'],
        image: image_url,
        published_at: @volume_info ['publishedDate',
        title: @volume_info ['title'],
        publisher: @volume_info ['publisher'],
      )
    end
  
    def new_from_id(google_books_api_id)
      url=url_of_creating_from_id(Google_books_api_id)
      item=get_json_from_url(url)
      new_from_item(item)
    end
 

# Where the error occurred ↓
# Use the following methods on the controller:
# The "keyword" sent as a parameter from the front end.
# as an argument.

    def search(keyword)
      # Convert requests from the front end to uri
      # The url_of_searching_from_keyword method is defined in the module below.
      url=url_of_searching_from_keyword(keyword)
      # Write to Log
      Rails.logger.debug(url)

      # Encoding url
      enc=Addressable::URI.encode(url)
      # Write to Log
      Rails.logger.debug(enc)

      Convert to #uri
      uri = URI.parse(enc)
      # Write to Log
      Rails.logger.debug(uri)
      
      # Output host and port number to terminal
      # putsuuri.host=>www.googleapis.com
      # puturi.port=>443

      require'net/https'
      http=Net::HTTP.new(uri.host,uri.port)
      Rails.logger.debug(http)
      http.use_ssl=true
      http.verify_mode=OpenSSL::SSL::VERIFY_NONE

  
      request=Net::HTTP::Get.new(uri.request_uri)
      Write to log whether a x#get request has been made
      Correction # Write the log to see if an object has been created to request get
      Rails.logger.debug(request)
  
  # ↓ Something is wrong here (I can't get a response)
      response = http.request(request)
  # ↑ I want the process to work so far

lib/google_books_api.rb


module GoogleBooksApi
  
  default_of_searching_from_keyword(keyword)
    "https://www.googleapis.com/books/v1/volumes?q=#{keyword}&country=JP"
  end

Tried

httpFrom http to https communication
Initially
response=Net::HTTP.get(uri)
I used the code, but I made it ssl as shown in the model above.

R Determine where the problem occurs when debugging by Rails.logger.debug

Decompose the code and see how far Rails.logger.debug is running

In the log of Production.log,
Net::HTTP::Get:0x00007f7e4af86658
I found the log that Rails sent me a get request to Google Books Api itself.

Supplementary information (for example, FW/Tool Version)

We are still investigating the cause of the error, but we are asking you this question because we want to know other people's wisdom and how to deal with it (debug).
Please help us all.
If you have any missing information or comments, please let me know!

Ruby 3.0.2
Rails 6.1.6

ruby-on-rails ruby aws

2022-09-30 22:06

1 Answers

The network mode on the Rails side was set to awsvpc according to the official configuration example, but
https://docs.aws.amazon.com/ja_jp/AmazonECS/latest/bestpracticesguide/networking-outbound.html
According to ↑'s article, awsvpc cannot use internet gateway.
Image Description
Therefore, in the task definition on the Rails side, we re-created the network mode with bridge as follows.
Image Description
As a result, the aws console received an error message stating that the service could not be created because the port was duplicated, and the service could not be created.
After reading some articles
https://aws.amazon.com/jp/premiumsupport/knowledge-center/dynamic-port-mapping-ecs/
Why don't we discover and do dynamic port mapping?I realized.
Image Description
Then, according to the article, we re-created the network area.
The remade one is
● Application Load Balancer for Backend
● Target Group (for backend)
●Rails side task definition
● Rails side services
Yes.
Then, because I remade the load balancer, I reconfigured Route 53 (A record).
After that, I found out that the Rails security group needs to add the above load balancer to the inbound rule, and I edited the Rails security group inbound rule according to the following.
Image Description
I can't show you all the security, so I'm going to take an excerpt as follows.
Image Description
Do the above and
I entered the Rails side production url into the browser and accessed it, and the browser now has access to health check actions that simply return head200 (white screen appears).
Also, after making an ssh connection to the cluster, we broke into the Rails side container with docker exec and pinged and curl commands to the outside (GoogleApi), and confirmed that the response was successful.
However, another error occurred in the connection with Rds, which was operating normally with network mode set to awsvpc, and Nxt and Rails were unable to communicate.
After reading several articles, I found the following articles and
https://qiita.com/fkana/items/a4b3c4d5d8ca27cd20ec
https://zatoima.github.io/aws-ec2-psql-install.html
I noticed that the security on the Rails side is suspicious.
We then set the following inbound rules for the security group on the Rails side:
Image Description
As a result, the title
"I want to work with Net::OpenTimeout (execution expired) in Net::HTTP"
The goal has been achieved.
However, I have no knowledge of aws configuration or security.
As the above image has a wide range of access, it is likely to be a temporary measure due to security issues (do I need to narrow down the range of access?).


2022-09-30 22:06

If you have any answers or tips


© 2024 OneMinuteCode. All rights reserved.