DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Priority
Acknowledgment is made of applicant’s claim for foreign priority under 35 U.S.C. 119 (a)-(d). The certified copy has been filed in parent Application No. EP23165685, filed on 2023.03.30.
Should applicant desire to obtain the benefit of foreign priority under 35 U.S.C. 119(a)-(d) prior to declaration of an interference, a certified English translation of the foreign application must be submitted in reply to this action. 37 CFR 41.154(b) and 41.202(e).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-16 is/are rejected under 35 U.S.C. 103 as being unpatentable over Szakelyhidi et al. (US 20130088058 A1, hereinafter Szakelyhidi), in view of Faaborg et al. (US 20230026575A1, hereinafter Faaborg), further in view of National Digital CarSeat Form (“National Digital Car Seat Check Form (NDCF) Digital Form User Guide”, 202010, hereinafter NDCF).
Regarding Claim 14, Szakelyhidi teaches a system of [[vision-based]] child seat mounting assistance in a vehicle, the system implementing a method including:; detecting an act of installing a child seat in the vehicle [[based on at least one image of an interior of the vehicle]]; [[determining a child seat model type of the child seat to be mounted in the vehicle; and]] outputting, on a display, an assistance for mounting the child seat in the vehicle [[ based on a three-dimensional reference model of the determined child seat model type ]] (Szakelyhidi et al., Paragraph [0011], "Provided is a child car seat that includes: a seat base secured to a seat of a vehicle; an infant carrier removably connected to the seat base; and an interface device coupled to at least one of the seat base or the infant carrier and configured to provide an indication to a user that the seat base is properly secured to the seat of the vehicle."; Paragraph [0012], "The interface device may be configured to provide the indication to the user when the infant carrier is connected to the seat base. The interface device may include a display providing a visual indication to the user that the seat base is properly secured to the seat.")
But, Szakelyhidi does not explicitly disclose that the assistance are vision-based based on at least one image of an interior of the vehicle; determining a child seat model type of the child seat to be mounted in the vehicle; and … . based on a three-dimensional reference model of the determined child seat model type
However, Faaborg teaches detecting an act of installing a object based on at least one image of an object (Faaborg et al., Paragraph [0003], "In general aspect, a computer-implemented method includes receiving a two-dimensional (2-D) image of a scene captured by a camera <read on at least one image of an interior of the vehicle>, and recognizing one or more objects in the scene depicted in the 2-D image <read on recognizing a child seat in the vehicle interior>."; "The method further includes overlaying augmented reality content over a display of the 2-D image of the scene using the depth from the camera of the at least one recognized object having known real-world dimensions to position the augmented reality content.") outputting, on a display, an assistance for mounting the object based on a three-dimensional reference model of the determined object type (Faaborg et al., Paragraph [0002], "Augmented reality (AR) applications add virtual components or objects (e.g., three-dimensional (3-D) AR objects) to images of a real-world scene. An AR application may, for example, add the virtual components to camera images on a smartphone display to enhance a user's view of the real-world scene <read on outputting, on a display, an assistance for mounting the child seat in the vehicle based on a three-dimensional reference model of the determined child seat model type>.")
Faaborg and Szakelyhidi are analogous since both of them are dealing with providing feedback or guidance about real-world physical objects through an electronic interface. Szakelyhidi provided a way of automatically leveling and tensioning a child car seat base, sensing installation conditions (such as seat level, belt tension, and harness position) and providing an interface indication that the seat base is properly secured to the vehicle seat. Faaborg provided a way of receiving a camera image, recognizing objects, estimating depth from known real-world object dimensions, and overlaying three-dimensional augmented reality content on the displayed image of the scene to enhance the user's understanding of the real-world arrangement. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to incorporate the depth-aware augmented reality overlay pipeline of taught by Faaborg the modified invention of Szakelyhidi such that, instead of only indicating that the CRS is properly installed, the system overlays a three-dimensional reference model of the seat on a live camera image of the vehicle interior to visually assist the caregiver while installing the child seat. The motivation is to reduce known high rates of child seat misuse and to provide clearer, more intuitive visual guidance by using augmented reality techniques that add three-dimensional virtual objects to camera images to enhance a user's view of the real-world scene as taught by Faaborg.
But the combination does not it does not explicitly disclose that the recognized objects are child seats or that the three-dimensional AR content is a reference model selected for a specific child seat model type.
However, NDCF teaches determining a child seat model type of the child seat to be mounted in the vehicle (NDCF Digital CPS User Guide, "Step 3 of 5: Findings on Arrival," items 26-28: " 26. Model Name Document the model name of the car seat being used on arrival. If the model name cannot be determined, enter “U” in the text box. 27. Model Number Document the model number of the car seat being used on arrival. If the model number cannot be determined, enter “U” in the text box. 28. MFR Date Document the date of manufacture of the car seat being used on arrival")
NDCF and Szakelyhidi are analogous since both address proper selection and installation of child restraint systems in vehicles based on specific seat models. Szakelyhidi provided a child car seat and base with sensors, control logic, and a user interface for automatically leveling and tensioning the CRS and indicating to the user when the seat base is properly secured to the vehicle seat. NDCF provided a way of systematically capturing car seat manufacturer, model name, and model number information via a user interface so that installation checks, expiration validation, and use guidance can be tailored to the specific child seat model in use. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to incorporate the explicit capture and use of car seat manufacturer/model/model number taught by NDCF into modified invention of Szakelyhidi such that the system determines a child seat model type (for example, by manufacturer and model name/number) and then uses that type to select an appropriate three-dimensional reference model for installation assistance. The motivation is to provide more accurate, seat-specific guidance, including verifying that the seat is not expired and applying model-specific installation checks, thereby improving safety and compliance with manufacturer instructions as described in "Step 3 of 5: Findings on Arrival" of the NDCF guide.
Regarding Claim 1, it recites limitations similar in scope to the limitations of Claim 14 but as a method and the combination of Szakelyhidi, Faaborg and NDCF teaches all the limitations as of Claim 14. Therefore is rejected under the same rationale.
Regarding Claim 2, the combination of Szakelyhidi, Faaborg and NDCF teaches the invention in Claim 1.
The combination further teaches comprising at least one of: [[applying an object detection algorithm to detect the child seat in the at least one image;]] [[determining depth information of the vehicle's interior;]] [[obtaining one or more door state signals from the vehicle;]] or [[obtaining seat regions and/or doorway regions for the vehicle,]] wherein detecting the act of installing the child seat is based on at least one of the information provided by the object detection algorithm, the depth information, the one or more door state signals, and the seat regions and doorway regions. (Szakelyhidi, Paragraph [0011], "Provided is a child car seat that includes: a seat base secured to a seat of a vehicle; an infant carrier removably connected to the seat base; and an interface device ... configured to provide an indication to a user that the seat base is properly secured to the seat of the vehicle."; Paragraph (0012], "The interface device may ... include a display providing a visual indication to the user that the seat base is properly secured to the seat.")
But, Szakelyhidi does not disclose image-based object detection, depth information, door-state signals, or region masks for the vehicle. Accordingly, all the specific "object detection algorithm," "depth information," "door state signals," and "seat/doorway regions".
However, Faaborg teaches at least: applying an object detection algorithm to detect the child seat in the at least one image; and determining depth information of the vehicle's interior (Faaborg, Paragraph (0003], "In general aspect, a computer-implemented method includes receiving a two-dimensional (2-0) image of a scene captured by a camera, and recognizing one or more objects in the scene depicted in the 2-0 image <read on applying an object detection algorithm to detect the child seat in the at least one image>. The method also includes determining whether the one or more recognized objects have known real-world dimensions, and determining a depth from the camera of at least one recognized object having known realworld dimensions <read on determining depth information of the vehicle's interior in the region of the child seat>.")
Faaborg and Szakelyhidi are analogous since both address providing guidance about realworld objects using electronic systems: Szakelyhidi provided a way of detecting proper child-seat installation using onboard sensors and notifying the user via a display. Faaborg provided a way of automatically analyzing a camera image using object detection and depth computation. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to incorporate the augment the sensor-based installation detection with a vision pipeline tauight by Faaborg into the invention of automatically analyzing a camera image using object detection such that detecting the act of installing the child seat is based on applying an object detection algorithm to detect the child seat in camera images and determining depth information of the vehicle interior near the child seat. The motivation is to improve robustness and flexibility of installation detection by combining visual information with seat-mounted sensors so that the system can detect installation events and positions even when some sensors are missing or misconfigured.
Regarding Claim 3, the combination of Szakelyhidi, Faaborg and NDCF teaches the invention in Claim 1.
The combination further teaches determining the child seat model includes: displaying, at the display, [[ a selection option for selecting the child seat model type of the child seat that is to be mounted in the vehicle; and receiving an input of the selected child seat model type.]] (Szakelyhidi, Paragraph [0011], "Provided is a child car seat that includes: a seat base secured to a seat of a vehicle; an infant carrier removably connected to the seat base; and an interface device coupled to at least one of the seat base or the infant carrier and configured to provide an indication to a user that the seat base is properly secured to the seat of the vehicle."; Paragraph [0012], "The interface device may be configured to provide the indication to the user when the infant carrier is connected to the seat base. The interface device may include a display providing a visual indication to the user that the seat base is properly secured to the seat.")
But Szakelyhidi does not explicitly taught a selection option for selecting the child seat model type of the child seat that is to be mounted in the vehicle; and receiving an input of the selected child seat model type.
However, NDCF teaches determining the child seat model includes: displaying, at the display, a selection option for selecting the child seat model type of the child seat that is to be mounted in the vehicle; and receiving an input of the selected child seat model type (NDCF Digital CPS User Guide v9.0, "Step 3 of 5: Findings on Arrival," items 26-28: "" items 26-28: " 26. Model Name Document the model name of the car seat being used on arrival. If the model name cannot be determined, enter “U” in the text box. 27. Model Number Document the model number of the car seat being used on arrival. If the model number cannot be determined, enter “U” in the text box. 28. MFR Date Document the date of manufacture of the car seat being used on arrival")
NDCF and Szakelyhidi are analogous since both address ensuring proper selection and installation of child restraint systems in vehicles. Szakelyhidi provides a child car seat and base with sensors, control logic, and a user interface that indicate when the seat base is properly secured to the vehicle seat, focusing on physical installation conditions (level, belt tension, harness position, etc.). NDCF provides a digital workflow and user interface for technicians to select and record the car seat manufacturer, model name, and model number of the seat being used on arrival so that downstream checks, expiration verification, and guidance are specific to that model. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to incorporate the explicit model-selection UI of NDCF into the installation-feedback system of Szakelyhidi such that the system, when determining the child seat model, displays on its interface a selection option to choose the child seat model type that is to be mounted in the vehicle and receives the user's input of the selected model type for use in further installation checks and guidance. The motivation is to provide model-specific safety verification and guidance, including confirming that the selected seat is appropriate, not expired, and subject to the correct installation criteria as taught by NDCF's "Step 3 of 5: Findings on Arrival" while leveraging Szakelyhidi's installed-sensor feedback capabilities to improve overall child restraint safety.
Regarding Claim 4, the combination of Szakelyhidi, Faaborg and NDCF teaches the invention in Claim 2.
The combination further teaches wherein determining the child seat model type includes: [[determining a list of child seat model types by comparing outputs of the object detection algorithm and/or the depth information with predefined designs of child seat models with respect to size and/or shape;]] and displaying, at the display, [[ the list of child seat model types.]] (Szakelyhidi, Paragraph [0011], "Provided is a child car seat that includes: a seat base secured to a seat of a vehicle; an infant carrier removably connected to the seat base; and an interface device ... configured to provide an indication to a user that the seat base is properly secured to the seat of the vehicle.")
Szakelyhidi does not explicity disclose but NDCF teaches determining a list of child seat model types by comparing outputs of the object detection algorithm and/or the depth information with predefined designs of child seat models with respect to size and/or shape; and displaying, at the display, the list of child seat model types (NDCF Digital CPS User Guide, "Step 3 of 5: Findings on Arrival," items 26-28: "" items 26-28: " 26. Model Name Document the model name of the car seat being used on arrival. If the model name cannot be determined, enter “U” in the text box. 27. Model Number Document the model number of the car seat being used on arrival. If the model number cannot be determined, enter “U” in the text box. 28. MFR Date Document the date of manufacture of the car seat being used on arrival")
NDCF and Szakelyhidi are analogous because both are concerned with proper use and installation of child seats in vehicles; Szakelyhidi provides a child car-seat installation system with sensors and a display for indicating proper installation, NDCF provides a structured user interface for identifying the specific child seat model in use by selecting from a list of manufacturer/model entries stored in a database. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to incorporate the model-selection interface of NDCF into the modified invention of Szakelyhidi such that determining the child-seat model type includes maintaining a list of child-seat model entries and displaying that list on the interface for the user to select the appropriate model. The motivation is to ensure that installation guidance and checks are tailored to the correct model of child seat, as different models have specific installation requirements, expiration dates, and manufacturer instructions.
Regarding Claim 5, the combination of Szakelyhidi, Faaborg and NDCF teaches the invention in Claim 1.
The combination further teaches comprising receiving the three-dimensional reference model (Faaborg, Paragraph [0002], "Augmented reality applications add virtual components or objects (e.g., three-dimensional (3-0) AR objects) <read on three-dimensional reference model> to images of a real-world scene."; Paragraph [0003], "The method further includes overlaying augmented reality content over a display of the 2-D image of the scene")
Faaborg and Szakelyhidi are analogous since both address providing guidance about realworld objects using electronic systems. Szakelyhidi provided the child-seat installation context. Faaborg provided a way of using 3D AR content matched to recognized object types. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to incorporate stored or retrieved three-dimensional object reference models taught by Faaborg into modified invention of Szakelyhidi such that system will be able to give users an accurate 3D model of their specific seat, improving clarity of installation guidance.
The combination further teaches receiving data according to the determined child seat model type (NDCF Digital CPS User Guide, Step 3 of 5, items 26-28: "26. Car Seat MFR Select the manufacturer name of the car seat ... 27. Model Name Select the model name ... 28. Model Number Document the model number ... "; items 32-33: "For the most current recall link, visit NHTSA.gov. Internet connection required." This shows that the determined model is used to access remote, model-specific data)
NDCF and Szakelyhidi are analogous since both address child-seat identification and correct use. Szakelyhidi provided a system that guides installation, while NDCF provided a workflow that uses a selected car-seat model to retrieve model-specific remote data (e.g., recalls). Therefore, it would have been obvious to one of ord inary skill in the art before the effective filing date of the claimed invention was made to incorporate reading car seat related data from remote stored location taught by NDCF into modified invention of Szakelyhidi so that after determining the child seat model type, the system retrieves additional model-specific data from a remote database, such as a manufacturer server, and uses this data for installation assistance. The motivation is to ensure model specific accuracy, just as NDCF uses model identification to access recall information through remote servers.
Regarding Claim 6, the combination of Szakelyhidi, Faaborg and NDCF teaches the invention in Claim 5.
The combination further teaches [[ storing ]] the determined [[ child seat ]] model type and the three-dimensional reference model [[ in a database of the vehicle ]] (Faaborg, Paragraph (0002], "three-dimensional (3-0) AR objects"; Paragraph (0003], "overlaying augmented reality content").
Faaborg and Szakelyhidi are analogous since both address providing guidance about realworld objects using electronic systems. Szakelyhidi provided a way of supplied the child-seat context. Faaborg taught AR content that must be stored and retrieved; Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to incorporate to store the 30 reference model associated with a child-seat type inside a vehicle database after obtaining it taught by Faaborg into modified invention of Szakelyhidi such that system will be able to nable future reuse without re-downloading.
The motivation is improved reliability, offline availability, and reduced latency. Szakelyhidi does not explicitly disclose storing the determined child seat model type in a database of the vehicle.
However, NDCF teaches storing the determined child seat model type and reference model in a database of the vehicle (NDCF Digital CPS User Guide, Step 3 items 26-30: manufacturer, model name, model number, expiration, recall status stored as structured fields)
NDCF and Szakelyhidi are analogous since both address child-seat identification and correct use. Szakelyhidi provided a system that guides installation, NDCF provided a persistent recording of car-seat model information, while Szakelyhidi provided the child-seat installation system with a controller and interface. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to incorporate persistent fields taught by NDCF into modified invention of Szakelyhidi such that the determined child-seat model type is stored in a vehicle database for future use. The motivation is to avoid repeated re-entry and to enable model-specific guidance on future installations.
Regarding Claim 7, the combination of Szakelyhidi, Faaborg and NDCF teaches the invention in Claim 6.
The combination further teaches [[ storing ]] one or more images of the interior of the vehicle [[ in the database of the vehicle]]. (Szakelyhidi, Paragraph [0011], "a seat base secured to a seat of a vehicle ... "; Paragraph [0012], "the interface device may include a display providing a visual indication to the user that the seat base is properly secured to the seat of the vehicle.")
But, Szakelyhidi does not explicitly disclose storing images.
However, Faaborg teaches storing one or more images of the objects in the database (Faaborg, Paragraph [0003], "receiving a two-dimensional (2-D) image of a scene captured by a camera <read on receiving one or more images> ... ").
Faaborg and Szakelyhidi are analogous since both involve systems that assist users in interacting with real-world physical vehicle-related objects. Szakelyhidi provided the child-seat installation system inside the interior of a vehicle. Faaborg provided a way of capturing images from a camera and using them for object recognition and AR processing. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to incorporate the image-capture pipeline taught by Faaborg into the modified invention of Szakelyhidi such that images of the vehicle interior are captured and then stored in a vehicle database for installation verification, documentation, or AR alignment. The motivation is to maintain a record of interior conditions, enable re-checking of installation, and support AR overlays without recapturing images.
Regarding Claim 8, the combination of Szakelyhidi, Faaborg and NDCF teaches the invention in Claim 2.
The combination further teaches wherein determining the child seat model type includes: [[determining whether the child seat to be mounted corresponds to a child seat model type stored in a database of the vehicle by comparing outputs of the object detection algorithm and/or the depth information with data stored in the database;]] and [[ in response to determining that the child seat to be mounted corresponds to a child seat model type stored in the database, determining the respective child seat model type.]] (Szakelyhidi, Paragraphs [0011] "Provided is a child car seat that includes: a seat base secured to a seat of a vehicle; an infant carrier removably connected to the seat base; and an interface device ... configured to provide an indication to a user that the seat base is properly secured to the seat of the vehicle."; Paragraph [0012], "The interface device may ... include a display providing a visual indication to the user that the seat base is properly secured to the seat.")
Szakelyhidi does not explicitly teach determining whether the child seat to be mounted corresponds to a child seat model type stored in a database of the vehicle by comparing outputs of the object detection algorithm and/or the the depth information with data stored in the database; and in response to determining that the child seat to be mounted corresponds to a child seat model type stored in the database, determining the respective child seat model type.
However, Faaborg teaches determining whether the object corresponds to a model type stored in a database of by comparing outputs of the object detection algorithm and/or the depth information with data stored in the database (Faaborg, Paragraph [0003], "The method also includes determining whether the one or more recognized objects have known real-world dimensions, and determining a depth from the camera of at least one recognized object having known real-world dimensions.")
Faaborg and Szakelyhidi are analogous since both are concerned with understanding and guiding interactions with objects in a real-world environment using electronic systems. Szakelyhidi provides a child car seat and base with sensors, control logic, and a display interface configured to indicate when the seat base is properly secured to the vehicle seat. Faaborg provided a way of comparing recognized objects in a camera image against stored "known real-world dimensions" in a knowledge database to determine whether they match a known type and to derive depth. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to incorporated vision pipeline taught by Faaborg into modify the childseat installation system of Szakelyhidi such that determining the child seat model type includes determining whether the child seat to be mounted corresponds to one of a plurality of stored models by comparing output from an object detection algorithm and/or depth information derived from the vehicle interior with data stored in a database. The motivation is to increase automation and robustness of model identification by using camera-based recognition and depth information in addition to sensor-based checks, thereby improving safety and reducing the chance of using an incorrect or unapproved seat model.
Szakelyhidi does not explicitly disclose but NDCF teaches in response to determining that the child seat to be mounted corresponds to a child seat model type stored in the database, determining the respective child seat model type (NDCF Digital CPS User Guide, "Step 3 of 5: Findings on Arrival," " items 26-28: " 26. Model Name Document the model name of the car seat being used on arrival. If the model name cannot be determined, enter “U” in the text box. 27. Model Number Document the model number of the car seat being used on arrival. If the model number cannot be determined, enter “U” in the text box. 28. MFR Date Document the date of manufacture of the car seat being used on arrival”)
NDCF and Szakelyhidi are analogous since both address proper selection and installation of child restraint systems in vehicles. Szakelyhidi provided the child-seat installation hardware and interface through which installation guidance and status indications are delivered. NDCF provided a way of organizing child seat model information in a database and using that data to identify the particular model in use (by manufacturer, model name, and model number). Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to incorporate a similar database of child seat models into the installation assistance system of Szakelyhidi such that, once the system determines that the detected child seat corresponds to data stored in the database, the system determines the respective child seat model type from that database entry for use in subsequent guidance and checks. The motivation is to ensure that installation assistance and safety checks are tailored to the exact model of child seat, using stored model-specific data as taught by the NDCF workflow, thereby improving safety, documentation, and compliance with manufacturer instructions.
Regarding Claim 9, the combination of Szakelyhidi, Faaborg and NDCF teaches the invention in Claim 8.
The combination further teaches receiving the three-dimensional reference model from the database [[ of the vehicle ]] (Faaborg, Paragraph [0002], "Augmented reality (AR) applications add virtual components or objects (e.g., three-dimensional (3-D) AR objects) to images of a real-world scene. An AR application may, for example, add the virtual components to camera images on a smartphone display to enhance a user's view of the real-world scene."; Paragraph [0003], "The method further includes overlaying augmented reality content over a display of the 2-D image of the scene using the depth from the camera of the at least one recognized object having known real-world dimensions to position the ugmented reality content." [0037], "Object recognizer module 41 may search knowledge database 50 (e.g., by object name or by object picture) to acquire available information on the dimensions of each of the recognized objects").
Faaborg and Szakelyhidi are analogous since both of them are dealing with systems that use electronic components to help a user understand and correctly interact with real-world objects. Szakelyhidi provided a way of guiding installation of a child car seat using sensors, a controller, and an interface device with a display that indicates when the seat base is properly secured to the vehicle seat. Faaborg provided a way of using a camera image, recognizing objects, consulting stored information (including known real-world dimensions), and overlaying three-dimensional augmented reality content onto the displayed image, which implies that three-dimensional AR objects are stored and retrieved when needed to represent recognized object types. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to incorporate the concept of storing and retrieving three-dimensional AR content taught by Faaborg into modified invention of Szakelyhidi such that, once the system has determined the relevant child seat model type, it receives the corresponding three-dimensional reference model from a database and uses it for the mounting assistance and its dependents. The motivation is to enable model-specific AR guidance by maintaining a stored three dimensional representation of each supported child seat and retrieving the appropriate model when needed, improving the clarity and precision of installation instructions beyond simple status indicators.
NDCF further teaches receiving the three-dimensional reference model from the database of the vehicle (NDCF, Page 2, "' Sign into the app, connect to Wi-Fi and begin downloading stored car seat checks").
The NDCF and Szakelyhidi are analogous since both address proper selection and installation of child restraint systems in vehicles. Szakelyhidi provided a child car seat and base with sensors and an interface device that indicates whether the seat base is properly secured to the vehicle seat. The NDCF guide provided a way of organizing child seat information in a structured record system (by manufacturer, model name, and model number) and having the user identify which specific model is in use so that subsequent checks and guidance can be tailored to that model. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to incorporate the model-based record-keeping of NDCF into the installation assistance system of Szakelyhidi such that the vehicle system maintains a database of child seat model entries and, after determining which model is present (as in Claim 8), receives the corresponding three-dimensional reference model from the vehicle's database for use in the mounting assistance. The motivation is to provide seat-specific installation guidance and verification by associating each stored database record with its own three-dimensional reference model, thereby improving safety, traceability, and compliance with model-specific manufacturer instructions.
Regarding Claim 10, the combination of Szakelyhidi, Faaborg and NDCF teaches the invention in Claim 1.
The combination further teaches the assistance for mounting the child seat includes: displaying, at the display, a projection of the three-dimensional reference model being correctly mounted [[ on a current two-dimensional camera image]] of the vehicle's interior. (Szakelyhidi, Paragraph [0011], "Provided is a child car seat that includes: a seat base secured to a seat of a vehicle; an infant carrier removably connected to the seat base; and an interface device ... configured to provide an indication to a user that the seat base is properly secured to the seat of the vehicle."; Paragraph [0012], "The interface device may include a display providing a visual indication to the user that the seat base is properly secured to the seat.")
But Szakelyhidi does not disclose projecting a three-dimensional reference model into a current two-dimensional camera image.
However, Faaborg teaches displaying, at the display, a projection of the three-dimensional reference model being correctly mounted on a current two-dimensional camera image (Faaborg, Paragraph [0003], "In general aspect, a computer-implemented method includes receiving a two-dimensional (2-0) image of a scene captured by a camera <read on current two dimensional camera image of the vehicle's interior>, and recognizing one or more objects in the scene depicted in the 2-0 image."; Paragraph [0003], "The method further includes overlaying augmented reality content over a display of the 2-0 image of the scene using the depth from the camera of the at least one recognized object having known real-world dimensions to position the augmented reality content <read on displaying, at the display, a projection of the three-dimension... in the current two-dimensional camera image>.")
Faaborg and Szakelyhidi are analogous since both involve providing guidance about physical objects using an electronic display: Szakelyhidi provided a way of indicating proper installation of a child car seat via an interface device and display, and Faaborg provided a way of overlaying three dimensional augmented reality content on a two-dimensional camera image to enhance the user's understanding of a real-world scene. Szakelyhidi provides a way of sensing installation conditions and indicating when the seat base is properly secured; Faaborg provides a way of obtaining a camera image, computing depth for recognized objects, and overlaying 3D AR content on that image in real time. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to incorporate the AR overlay approach taught by Faaborg into the installation-assistance display of Szakelyhidi such that the assistance for mounting includes displaying, at the display, a projection of the three-dimensional reference model in the current two dimensional camera image of the vehicle interior instead of only showing simple status indications. The motivation is to improve clarity and usability of installation instructions by visually aligning a 3D reference of the correct seat placement with the actual camera view of the vehicle interior, thereby reducing mis installation and improving child safety.
Regarding Claim 11, the combination of Szakelyhidi, Faaborg and NDCF teaches the invention of assistance for mounting the child seat in Claim 2.
The combination further teaches the assistance for mounting the child seat includes: generating a two-and-a-half-dimensional model of the [[ child seat to be mounted )] object by aggregating the information provided by the object detection algorithm and the depth information; and displaying, at the display, a projection of the three-dimensional reference model being correctly mounted and the two-and-a-half-dimensional model of the [[ child seat to be mounted ]] object on a rendered three-dimensional visualization of a camera image [[ of the vehicle's interior]). (Faaborg, Paragraph [0003], "In general aspect, a computer-implemented method includes receiving a two-dimensional (2-D) image of a scene captured by a camera, and recognizing one or more objects in the scene depicted in the 2-D image. The method also includes determining whether the one or more recognized objects have known real-world dimensions, and determining a depth from the camera of at least one recognized object having known real-world dimensions."; "The method further includes overlaying augmented reality content over a display of the 2-D image of the scene using the depth from the camera of the at least one recognized object having known real-world dimensions to position the augmented reality content."; it is two-and-a-half-dimensional model," it effectively constructs a depth-augmented representation of objects in a 2D image (a classic 2.5D concept) and projects 3D content into that image accordingly)
Faaborg and Szakelyhidi are analogous: both deal with using sensing and computation to help a user understand and interact with a real-world arrangement. Szakelyhidi provides the child-seat installation context and the idea of using a display and sensors to guide proper installation. Faaborg provides the computer-vision pipeline that turns a 2D camera image plus depth computation into a 2.50 representation used to correctly place 3D AR content. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to incorporate using a camera-based AR pipeline taught by Faaborg into the modified invention of Szakelyhidi such that the system generates a depth-augmented (2.50) model of the detected child seat in the vehicle interior from object detection and depth information, and displays a projection of a 3D child-seat reference model over the 2D camera view of the interior to guide installation. The motivation is to give more intuitive, spatially accurate guidance than purely textual or simple visual indicators, reducing mis installation by visually showing how the seat should align in the real vehicle scene.
Regarding Claim 12, the combination of Szakelyhidi, Faaborg and NDCF teaches the invention in Claim 1.
The combination further teaches wherein the display includes [[at least one of augmented reality glasses and ]] a vehicle's main display. (Szakelyhidi, Paragraph [0012], "The interface device may include a display providing a visual indication to the user that the seat base is properly secured to the seat.")
But Szakelyhidi does not explicitly disclose the display type as augmented reality glasses However, Faaborg teaches the display includes at least one of augmented reality glasses (Faaborg, Paragraph [0002], "An AR application may, for example, add the virtual components to camera images on a smartphone display to enhance a user's view of the real-world scene."; [0015], “The disclosed methods may be implemented for AR applications (e.g., mobile AR applications) using, for example, existing single camera configurations of devices (e.g., smartphones, head worn devices, or smartglasses”)
Szakelyhidi and Faaborg and analogous since both show that AR-style or status-style guidance can be presented on an electronic display. Szakelyhidi provided a way of guidance for proper installation of a child car seat via an interface device and display, and Faaborg provided a way of using augmented reality with smart glasses for overlaying three-dimensional augmented reality content on a two-dimensional camera image to enhance the user's understanding of a real-world scene. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to incorporate smart glasses technology taught by Faaborg into the invention of Szakelyhidi such that displays can take many forms (handheld device, embedded vehicle screen, or head-mounted AR device). It would have been obvious, in view of the flexibility of AR hardware platforms, to implement the claimed AR-based mounting assistance on either a vehicle's main display or on AR glasses to provide the user with a more integrated or handsfree experience.
Regarding Claim 13, the combination of Szakelyhidi, Faaborg and NDCF teaches the invention in Claim 1.
The combination further teaches the assistance for mounting the child seat includes [[outputting an audio signal]] leading to a correct mounting of the child seat based on a target position of the three-dimensional child seat model and a current position of the child seat to be mounted (Szakelyhidi, Paragraph [0012], "The interface device may include a display providing a visual indication to the user that the seat base is properly secured to the seat.")
But Szakelyhidi does not explicitly disclose outputting an audio signal.
However, Faaborg teaches the assistance ... includes outputting an audio signal ... based on a target position of the three-dimensional object and a current position of the object (Faaborg, Paragraph [0003], "The method further includes overlaying augmented reality content over a display of the 2-D image of the scene using the depth from the camera of the at least one recognized object having known real-world dimensions to position the augmented reality content <read on determining a target position of the three-dimensional model relative to the current position of the recognized object>."; [0010],. “Augmented reality (AR) applications (e.g., mobile AR apps, gaming applications, etc.) may seek …AR objects (e.g., videos, audio,”)
Faaborg and Szakelyhidi are analogous since both involve providing guidance about physical objects using an electronic display: Szakelyhidi provided a way of using a child car seat with sensors and an interface device that provides an indication when the seat base is properly secured. The indication is visual in the quoted paragraph, but in such systems it is known that feedback can also be provided via audible or tactile outputs. Faaborg provided a way of calculating positions for 3D AR content relative to real objects based on depth, effectively providing a target position of a 3D model relative to the real object shown in the camera image. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to incorporate the position-based AR guidance taught by Faaborg into the modified invention of Szakelyhidi such that the system, in addition to visual AR overlays and/or visual status indicators, outputs an audio signal that guides the user toward aligning the current child-seat position with a target position derived from the 3D reference model and its desired placement in the vehicle. The motivation is to improve usability and safety by providing multimodal feedback (audio plus visual) so that caregivers can keep their eyes on the seat and vehicle environment while still receiving clear guidance as they adjust the seat into the correct position, particularly in situations where viewing the display continuously is impractical.
Regarding Claim 15, the combination of Szakelyhidi, Faaborg and NDCF teaches the invention in Claim 14.
The combination further teaches a vehicle comprising: a camera for capturing images (Faaborg, Paragraph [0003], "In general aspect, a computer-implemented method includes receiving a two-dimensional (2-D) image of a scene captured by a camera <read on a camera for capturing images in a vehicle>")
As explained in rejection of claim 14, the obviousness for combining of augmented reality technology of Faaborg into Szakelyhidi is provided above.
Regarding Claim 16, it recites limitations similar in scope to the limitations of claim 14 and the combination of Szakelyhidi, Faaborg and NDCF teaches all the limitations as of Claim 14. And Faaborg discloses these features can be implemented on a computer-readable storage medium (Faaborg, Paragraph [0051], [0052], "The memory 404 stores information within the computing device 400. In one implementation, the memory 404 is a volatile memory unit or units. In another implementation, the memory 404 is a non-volatile memory unit or units. The memory 404 may also be another form of computer-readable medium, such as a magnetic or optical disk." “The processor 402 can process instructions for execution within the computing device 400,”).
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
US 20230237814 A1 METHOD AND SYSTEM FOR DETECTING A TYPE OF SEAT OCCUPANCY
US 20220126772 A1 VISION-BASED AIRBAG ENABLEMENT
US 11217083 B1 Intelligent camera child detection system
US 20210206343 A1 Vehicle Occupancy-Monitoring System
US 20190033655 A1 DISPLAY CELL STRUCTURE AND DISPLAY DEVICE USING QUANTUM DOT
US 20180190020 A1 PREDICTIVE AUGMENTED REALITY ASSISTANCE SYSTEM
US 9495399 B1 Augmented reality model comparison and deviation detection
US 20150193972 A1 METHOD OF 3D MODELING
US 20090160232 A1 CHILD SEAT AND METHOD FOR MONITORING INSTALLATION OF THE CHILD SEAT
US 20050278097 A1 Child seat monitoring system and method for determining a type of child seat
Any inquiry concerning this communication or earlier communications from the examiner should be directed to YUJANG TSWEI whose telephone number is (571)272-6669. The examiner can normally be reached 8:30am-5:30pm EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Kent Chang can be reached on (571) 272-7667. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/YuJang Tswei/Primary Examiner, Art Unit 2614