DETAILED ACTION
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claims 1-20 are pending.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claim 19 is rejected under 35 U.S.C. 101 because the claimed invention is directed to non-statutory subject matter.
Claim 19 recites limitation “A computer program product, comprising a computer program or computer instructions stored in a non-transitory computer-readable storage medium, …”. The computer program product may comprise only the computer program or computer instructions and may not comprise a non-transitory computer-readable storage medium since the computer program or computer instructions may not always be stored in a non-transitory computer-readable storage medium. For example, the computer program or computer instructions may be stored in a transitory medium such as carrier waves. A computer program per se, is not directed to one of the statutory categories, Gottschalk v. Benson, 409 U.S. at 72, 175 USPQ at 676-77. See MPEP § 2106(I). To overcome the rejection, applicant may amend the claim as, e.g., “A computer program product, comprising a non-transitory computer-readable storage medium and a computer program or computer instructions stored in the non-transitory computer-readable storage medium, ...”.
Claim Rejections - 35 USC § 103
The following is a quotation of pre-AIA 35 U.S.C. 103(a) which forms the basis for all obviousness rejections set forth in this Office action:
(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are such that the subject matter as a whole would have been obvious at the time the invention was made to a person having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the manner in which the invention was made.
Claim(s) 1-5, 7 and 9-20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Earp et al (US20220277474A1) in view of Hsieh et al (US20220301222A1).
Regarding claims 1 and 17-19, Earp teaches a method for positioning, comprising:
acquiring a reference image displaying both a target to be positioned and at least one target reference object,
(Earp, "receiving a camera frame providing view of a section of the floor", [0041]; "measuring key distances between a visible object and the walls", [0006]; acquiring a reference image (camera frame) that captures a target to be positioned (visible object) alongside target reference objects (walls))
acquiring first position information about the target reference object within the reference image, and
(Earp, "defining a plurality of reference points", [0041]; Hsieh, "select a plurality of sets of corresponding feature points from the captured image and the most similar image", [0009]; Earp teaches extracting first position information by defining reference points in the image. Hsieh strengthens this by explicitly teaching extracting sets of corresponding feature points from the captured image to act as the reference objects)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to incorporate the teachings of Hsieh into the system or method of Earp in order to precisely extract and match exact feature point coordinates from the captured image, thereby improving the accuracy and reliability of the reference points used for subsequent mapping to the floor plan. The combination of Earp and Hsieh also teaches other enhanced capabilities.
The combination of Earp and Hsieh further teaches:
acquiring relative position information about the target to be positioned relative to the target reference object, in the reference image;
(Earp, "measuring key distances between a visible object and the walls", [0006]; determining the relative position information by measuring key distances between the target object and the reference objects within the captured image frame)
acquiring second position information about the target reference object in a pre-generated indoor image; and
(Earp, "receiving a detailed floor plan of the floor", "mapping the plurality of reference points to the floor plan", [0041]; obtaining a pre-generated indoor image (detailed floor plan) and acquiring second position information by mapping the defined reference points to their corresponding positions on the floor plan)
determining target position information about the target to be positioned in the indoor image according to the first position information, the relative position information, and the second position information.
(Earp, "inputting those into the floor plan, the system learns to place all future object locations within the floor plan.", [0006]; "geo-referencing the one or more object in the floor plan if the object is determined to be placed in the first area.", [0041]; Hsieh, "calculate a capturing position and a capturing pose parameter when the image capturing device obtaining the captured image according to a geometric relationship between the captured image and the most similar image, a geometric relationship between the most similar image and the nearest image, and the plurality of sets of corresponding feature point coordinates", [0010]; Earp teaches computing the object's final coordinates (geo-referencing) within the floor plan based on the mapped reference points and relative distance inputs. Hsieh explicitly teaches executing calculations combining corresponding feature point coordinates (first/second position info) and geometric relationships (relative position) to find an accurate coordinate location in the indoor model. Incorporating Hsieh's specific geometric calculation framework into Earp would ensure highly precise target position derivation on the floor plan)
Regarding claim 2, the combination of Earp and Hsieh teaches its/their respective base claim(s).
The combination further teaches the method of claim 1, wherein determining the target position information about the target to be positioned in the indoor image according to the first position information, the relative position information, and the second position information comprises:
determining the target position information about the target to be positioned in the indoor image through geometric calculation according to the first position information, the relative position information, and the second position information.
(Earp, "As the one or more dimensions of the floor plan are known, the geo-referencing of the object may be done by simple mathematical calculations.", [0072]; Hsieh, "calculate a capturing position and a capturing pose parameter when the image capturing device obtaining the captured image according to a geometric relationship between the captured image and the most similar image, a geometric relationship between the most similar image and the nearest image, and the plurality of sets of corresponding feature point coordinates", [0010]; Earp teaches using mathematical calculations combining dimensions and distances to determine final coordinates. Hsieh explicitly teaches calculating positions through geometric relationships utilizing coordinate data. Incorporating Hsieh's explicit geometric calculation methodology into Earp's floor plan mapping would precisely translate spatial distances into exact location coordinates)
Regarding claim 3, the combination of Earp and Hsieh teaches its/their respective base claim(s).
The combination further teaches the method of claim 2, wherein determining the target position information about the target to be positioned in the indoor image through geometric calculation according to the first position information, the relative position information, and the second position information comprises:
determining reference mapping information between the target reference object in the reference image and the target reference object in the indoor image according to the first position information and the second position information; and
determining the target position information about the target to be positioned in the indoor image through geometric calculation according to the first position information, the second position information, the reference mapping information, and the relative position information.
(Earp, "Georeferencing in this invention refers to the process of mapping the camera frame to the physical dimensions of a floor plan.", [0006]; Hsieh, "calculate a capturing position and a capturing pose parameter ... according to a geometric relationship", [0009]; Earp establishes robust reference mapping information between the image frame pixels and the indoor floor plan physical dimensions. Hsieh teaches evaluating positioning data by performing geometric calculations based on this established structural and corresponding feature coordinate relationship)
Regarding claim 4, the combination of Earp and Hsieh teaches its/their respective base claim(s).
The combination further teaches the method of claim 3, wherein determining the target position information about the target to be positioned in the indoor image through geometric calculation according to the first position information, the second position information, the reference mapping information, and the relative position information comprises:
determining first position coordinates of the target reference object according to the first position information, determining a relative position parameter which corresponds to the target reference object according to the relative position information, and determining second position coordinates of the target reference object according to the second position information; and
calculating the first position coordinates, the second position coordinates, the reference mapping information, and the relative position parameter through geometric calculation to obtain the target position information about the target to be positioned in the indoor image.
(Earp, "The X and Y coordinates value of the point 502 represent the distance from walls or the distance from the two sides of the floor plan", [0087]; Hsieh, "obtain a plurality of sets of corresponding feature point coordinates", [0009]; Earp teaches acquiring point coordinates mapped to floor plans alongside relative distance parameters. Hsieh details obtaining corresponding feature point coordinates and utilizing them in a geometric calculation. Combined, it is obvious to extract first and second position coordinates along with mapping parameters to geometrically calculate the precise target position)
Regarding claim 5, the combination of Earp and Hsieh teaches its/their respective base claim(s).
The combination further teaches the method of claim 1, further comprising:
further acquiring, from different angles, additional reference images each displaying both the target to be positioned and the target reference object, and further acquiring, from the additional reference images, additional pieces of first position information about the target reference object and additional pieces of relative position information about the target reference object relative to the target to be positioned;
further acquiring additional pieces of second position information about the target reference object from the pre-generated indoor image;
further determining additional pieces of target position information about the target to be positioned in the indoor image according to the additional pieces of first position information, the additional pieces of relative position information, and the additional pieces of second position information; and
acquiring resultant position information about the target to be positioned in the indoor image according to the obtained additional pieces of target position information and the target position information.
(Earp, "an entire camera fleet comprising more than one camera may be set up to capture the floor. Some areas of the floor may be covered by multiple cameras but from different angles. Their camera frames may overlap.", [0075]; acquiring additional reference images of the target from multiple cameras capturing overlapping areas from different angles, which inherently involves extracting additional position information and coordinating these multiple positioning calculations to accurately establish a unified resultant position)
Regarding claim 7, the combination of Earp and Hsieh teaches its/their respective base claim(s).
The combination further teaches the method of claim 1, wherein acquiring the reference image displaying both the target to be positioned and the at least one target reference object comprises:
identifying the at least one target reference object positioned within the same plane as the target to be positioned; and
photographing the target to be positioned and the at least one target reference object to obtain the reference image displaying both the target to be positioned and the target reference object.
(Earp, "selecting a first area from the section of the floor in which the object needs to be tracked", [0041]; "The axes of the first area refer to the slope of the floor or hallway as depicted in the camera. The floor itself can be seen sloping as a "z-axis" in the 2-D camera frame.", [0064]; identifying the reference points and target object across the physical floor surface itself (within the same plane), wherein the camera photographs that specific section to obtain the reference image displaying both features together)
Regarding claim 9, the combination of Earp and Hsieh teaches its/their respective base claim(s).
The combination further teaches the method of claim 1, wherein acquiring the relative position information about the target to be positioned relative to the target reference object in the reference image comprises:
acquiring pixel coordinate information about the target reference object and pixel coordinate information about the target to be positioned from the reference image; and
determining, according to the pixel coordinate information about the target reference object and the pixel coordinate information about the target to be positioned, relative position information about the target reference object relative to the target to be positioned.
(Earp, "Once the person may be visible in the camera frame, it will appear as a red dot with X and Y coordinates.", "It must be made sure to drag or input corresponding pixel coordinates in the camera frame as well.", [0070]; obtaining precise X and Y pixel coordinate information for tracked targets and reference points within the camera frame, and using this pixel coordinate data to establish parameters and relative position information within the defined area)
Regarding claim 10, the combination of Earp and Hsieh teaches its/their respective base claim(s).
The combination further teaches the method of claim 1, wherein the relative position information comprises at least one of:
a distance between the target reference object and the target to be positioned; or
a relative position of the target reference object relative to a target projection, wherein the target projection is a projection of the target to be positioned onto the target reference object.
(Earp, "By measuring key distances between a visible object and the walls", [0006]; "The X and Y coordinates value of the point 502 represent the distance from walls or the distance from the two sides of the floor plan", [0087]; relative position information is defined by measuring the physical distance between the target reference object (such as walls) and the target to be positioned)
Regarding claim 11, the combination of Earp and Hsieh teaches its/their respective base claim(s).
The combination further teaches the method of claim 1, wherein the first position information comprises at least one of:
physical coordinate information about the target reference object within the reference image; or
pixel coordinate information about the target reference object within the reference image.
(Earp, "Georeferencing in this invention refers to the process of mapping the camera frame to the physical dimensions of a floor plan.", [0006]; "It must be made sure to drag or input corresponding pixel coordinates in the camera frame as well.", [0070]; obtaining first position information using pixel coordinate information corresponding to the target reference objects in the camera frame (reference image))
Regarding claim 12, the combination of Earp and Hsieh teaches its/their respective base claim(s).
The combination further teaches the method of claim 1, wherein the second position information comprises at least one of:
physical coordinate information about the target reference object within the indoor image; or
pixel coordinate information about the target reference object within the indoor image.
(Earp, "The X and Y measurements may be inputted into the table for each of the points. This only works without needing to calculate anything if the floor plan is resized so that one pixel is equal to one square inch.", [0070]; acquiring second position information by mapping coordinate points onto the floor plan (indoor image) sized so that pixel coordinate information accurately mirrors physical coordinate information)
Regarding claim 13, the combination of Earp and Hsieh teaches its/their respective base claim(s).
The combination further teaches the method of claim 3, wherein the reference mapping information comprises at least one of:
scale relationship; or
projection relationship.
(Earp, "determines by what rate the pixel to inch correlation changes between the camera frame and the floor plan.", [0064]; Hsieh, " calculating a second essential matrix between the most similar image and the nearest image, and inversely inference a scale ratio from the second essential matrix ", [0070]; Earp teaches determining a pixel-to-inch scale correlation rate. Hsieh teaches calculating a scale ratio to properly correlate image relationships. Thus, both references teach mapping information comprising a scale relationship)
Regarding claim 14, the combination of Earp and Hsieh teaches its/their respective base claim(s).
The combination further teaches the method of claim 1, wherein before acquiring the reference image displaying both the target to be positioned and the at least one target reference object, the method further comprises:
selecting the at least one target reference object from the indoor image.
(Earp, "receiving a detailed floor plan of the floor; selecting a first area from the section of the floor in which the object needs to be tracked; defining a plurality of reference points", [0041]; receiving the indoor image (floor plan) first, and analyzing it to define and select the plurality of reference points before processing current camera frames for live tracking)
Regarding claim 15, the combination of Earp and Hsieh teaches its/their respective base claim(s).
The combination further teaches the method of claim 1, wherein the indoor image is generated based on:
acquiring an indoor position parameter of indoor space; and
generating the indoor image according to the indoor position parameter, wherein the indoor image corresponds to the indoor space.
(Hsieh, "obtain a building information modeling (BIM) model of a target area", "generate at least one virtual camera, and control the at least one virtual camera to obtain a plurality of virtual images in the BIM model", [0010]; acquiring indoor position parameters utilizing a BIM model of the space, and generating virtual indoor images corresponding directly to that indoor space. Incorporating Hsieh's BIM-based image generation techniques into Earp's floor plan methodology would provide highly accurate, parameterized pre-generated indoor maps)
Regarding claim 16, the combination of Earp and Hsieh teaches its/their respective base claim(s).
The combination further teaches the method of claim 15, wherein the indoor position parameter comprises at least one of:
a wall line position parameter;
a door position parameter;
a stand column position parameter;
a wall position parameter; or
a window position parameter.
(Earp, "simplified to display only walls, columns etc.", [0060]; Hsieh, "correctly treat pillars and windows of the model as important features", [0044]; Earp teaches the floor plan parameters include positions of walls and columns. Hsieh further teaches generating indoor parameters incorporating walls, pillars, and windows. Combining them obviously meets the inclusion of wall, column, and window position parameters)
Regarding claim 20, the combination of Earp and Hsieh teaches its/their respective base claim(s).
The combination further teaches the method of claim 9, wherein the relative position information comprises at least one of:
a distance between the target reference object and the target to be positioned; or
a relative position of the target reference object relative to a target projection, wherein the target projection is a projection of the target to be positioned onto the target reference object.
(Earp, "By measuring key distances between a visible object and the walls", [0006]; obtaining a measurement of distance between the target reference object (walls) and the target object, satisfying the relative position information limitation. Also see comments on claim 10)
Claim(s) 6 is/are rejected under 35 U.S.C. 103 as being unpatentable over Earp et al (US20220277474A1) in view of Hsieh et al (US20220301222A1) and further in view of Hu et al (US20120219185A1).
Regarding claim 6, the combination of Earp and Hsieh teaches its/their respective base claim(s).
The combination further teaches the method of claim 5, wherein acquiring resultant position information about the target to be positioned in the indoor image according to the obtained additional pieces of obtained target position information and the target position information comprises:
obtaining an average value of the obtained additional pieces of target position information and the target position information, to obtain the resultant position information about the target to be positioned in the indoor image.
(Earp, see comments on claim 5; Hu, "If there are three or more reference images, the determined location for the site in the target image can be based on minimising a measure of distance from the three or more epipolar lines produced for the three reference images.", [0013]; Hu teaches minimizing the measured distance across three or more location reference calculations to determine the final target location. Minimizing distance (least squares regression/minimization) mathematically operates to find the spatial "average value" of multiple divergent coordinate sets)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to incorporate Hu's calculation technique into Earp's fleet of multiple cameras in order to yield a centralized average position from the multiple target position information pieces. The combination of Earp, Hsieh and Hu also teaches other enhanced capabilities.
Claim(s) 8 is/are rejected under 35 U.S.C. 103 as being unpatentable over Earp et al (US20220277474A1) in view of Hsieh et al (US20220301222A1) and further in view of Sweet et al (US20180350093A1).
Regarding claim 8, the combination of Earp and Hsieh teaches its/their respective base claim(s).
The combination further teaches the method of claim 7, wherein photographing the target to be positioned and the at least one target reference object to obtain the reference image displaying both the target to be positioned and the target reference object comprises:
taking a front photograph of the target to be positioned and the at least one target reference object to obtain the reference image displaying both the target to be positioned and the target reference object.
(Earp, see comments on claim 7; Sweet, "label 305, which is directly in front of the user 205 may appear undistorted when reproduced in a captured image", [0033]; Sweet teaches capturing an image where the target (e.g., label 305) is directly in front of the user/imaging device to obtain an undistorted reproduction)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to incorporate Sweet's technique of taking a front photograph into Earp's target positioning method in order to minimize angular distortion of the target and reference objects, thereby ensuring highly accurate coordinate mapping within the reference image. The combination of Earp, Hsieh and Sweet also teaches other enhanced capabilities.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JIANXUN YANG whose telephone number is (571)272-9874. The examiner can normally be reached on MON-FRI: 8AM-5PM Pacific Time.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Amandeep Saini can be reached on (571)272-3382. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272- 1000.
/JIANXUN YANG/
Primary Examiner, Art Unit 2662 7/15/2026
Prosecution Insights