DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on April 16th, 2026, has been entered.
Response to Amendment
Claims 1-18 in the claim set filed December 12th, 2025, were pending for examination in the Application No. 18/470,782 filed September 20th, 2023. In the remarks and amendments received on April 16th, 2026, claims 1, 8, and 10-11 are amended. Accordingly, claims 1-18 are currently pending for examination in the application.
Response to Arguments
Applicant’s arguments filed April 16th, 2026, regarding the rejection(s) of the independent claim(s) have been fully considered but are moot because the arguments do not apply to the new combination of the references being used in the current rejection below.
Priority (Previously Presented)
Acknowledgment is made of applicant’s claim for foreign priority under 35 U.S.C. 119 (a)-(d). The certified copy has been filed as foreign Patent Application No. CN 202311153118.3, filed on September 7th, 2023.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claims 1, 5, 8-11, 15, and 18 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Boroushaki et al. (Boroushaki; US 2022/0168899 A1).
Regarding claim 1, Boroushaki discloses a recognition system, comprising:
processing circuitry (para(s). [0044], recite(s)
[0044] “The controller 30 may be any type of processor, microprocessor, computer, or another type of processing logic device capable of performing the operations described herein. …”
) configured to
acquire an original image of one or more objects (para(s). [0032], [0036], and [0063], recite(s)
[0032] “Embodiments described herein are directed toward a control system and method which locate a partially or fully occluded target object in an area of interest. The location of the occluded object may be determined using a combination of visual information from an image sensor and RF-based location information. …”
[0036] “In one example embodiment, the area of interest 4 may include a group (or “pile”) of objects which may be the same as or similar to the target object (that is, the target object may be one of a plurality of physically similar objects disposed in a group or pile). In other example embodiments, the obstruction may be an item different (e.g. physically different or a different type of item) from the target object. For example, if the target item has a spherical shape (e.g. a ball shape), the obstruction may be flat walls of a container in which the sphere is located (e.g. the obstruction may be one or more of: a container, structural support, frame, packaging in which the target object is located, a covering or housing, or any other hard of soft physical material that may occlude perception of the target object, for example, by an image sensor but through which radio frequency (RF) signals may pass. …”
[0063] “In one example implementation (described in greater detail below), a UR5e robot may be controlled based on visual information from a camera sensor, along with a customized RF localization system operating with one or more software radios for receiving an RFID signal from a tag. The visual information may include a color image (e.g. an RGB image) and a depth image, e.g., the camera may output RGB-D data. …”
, where an “image” of a “a group (or ‘pile’) of object” is an original image of one or more objects);
acquire positioning information of a target object included in the one or more objects, the positioning information including information about the target object’s position in physical space (para(s). [0054], recite(s)
[0054] “FIG. 3 is a flow diagram showing an embodiment of a method for determining the location of the target object in the area of interest, which method may correspond to location algorithm 41. …”
, where the “location of the target object” is positioning information of a target object’s position in physical space);
determine whether the one or more objects include one or more non-target objects by comparing a position of the one or more objects to the positioning information (para(s). [0035], [0044], [0075], [0079], and [0086], recite(s)
[0035] “In the example of FIG. 1, the target object 1 is occluded by a structure 2. That is, structure 2 obstructs a view of some or all of object 1 and thus is structure 2 is sometimes referred to herein as an obstruction). The obstruction may or may not be one or more objects having the same or a similar appearance to the target object (i.e., structure 2 may be one or more objects having a physically similar appearance to the target object).”
[0044] “…The data and/or control signals may, for example, activate the control system to locate a particular target object (e.g., one having a specific tag frequency and/or identifier) or to search the area of interest 4 in general for any occluded objects to be identified. A robot operation may then be performed after target objection location is achieved.”
[0075] “For example, the range locator 430 may determine whether or not the target object is occluded by comparing the RFID location of the target object with the location and depth of the extracted occluded regions. If there is an overlap between the RFID location and one or more of the occluded regions and the depth of the RFID location is greater than the depth of the one or more occluded regions, then the range locator 430 may determine that the target object is occluded by the one or more occluded regions. Example embodiments of these models and operations are discussed below.”
[0079] “At 615, an extraction operation may be performed which includes identifying and then extracting occluded regions 520 in the model-based representation of the area of interest. The occluded regions may be extracted from the extractor 420 as described herein. The occluded regions may correspond to soft or hard obstructions that are within the field of vision of the camera focused on the area of interest. The occluded regions may be identified, for example, by projecting the occluding surface(s), e.g., obstruction(s), on to the camera frustrum. Once identified, the occluded regions may be extracted and used as a basis for generating one or more trajectories by the trajectory calculator.”
[0086] “The state of the area of interest (S) may be expressed as the robot joint state (X.sup.R∈R.sup.6) and the RFID location may be expressed as p=(x.sub.p, y.sub.p, z.sub.p), also the occluded regions and/or other objects and features may be determined by coordinate location. …”
, where “determin[ing] whether or not the target object is occluded” by at least “compar[ing]… location[s]” is determining whether the one or more objects include one or more non-target objects (i.e., “other objects” and/or “occlud[ing]” “one or more objects having the same or a similar appearance to the target object”) by at least comparing position of the one or more objects (e.g., “location and depth of the extracted occluded regions”) to the positioning information (e.g., “location of the target object”));
extract a target region in the original image based on the positioning information such that the target region includes the target object (para(s). [0123-0124] and [0128], recite(s)
[0123] “At 810, at least one RF-based attention mask is generated and applied on data output from the vision sensor. In embodiments, the mask may be generated based upon information/data from RF signals (e.g. based upon RF measurements), and applied on visual information (e.g. RGB-D information).”
[0124] “In one embodiment, multiple attention masks may be generated based on the RF measurements and applied on the output 562 from the vison sensor. This may be accomplished, for example, using two main strategies (and at three layers). First, an RF-based binary mask may be generated by cropping RGB and depth heightmaps to an area around the location of the target object. The area 563 around the location of the target operation may be designated, for example, by a square of a size less than the image generated by the vision sensor. …”
[0128] “Generating the attention mask(s) 571 and 572 and the kernel 575 may allow the network 580 to focus on the vicinity of the target object …”
, where “crop[ping]” images to “an area around the location of the target object” is extracting a target region in the original image based on positioning information (e.g., “location”) such that the target region includes the target object (e.g., “area around… the target object” includes the “target object”)); and
perform a target object recognition operation to recognize the target object, wherein the target object recognition operation is based on the extracted target region (para(s). [0043] and [0073], recite(s)
[0043] “In the example of FIG. 1, the target object 1 is fully occluded by obstruction 2. The obstruction may be any type of object which allows RF signals to pass. As previously described, the objection may be a soft obstruction or a hard objection. The obstruction may be another object which is the same as or similar to the target object, when, for example, the objects are in a pile or a collection or arrangement. In another case, the obstruction may be a cover or housing or packaging, a frame or container, or another type of material or structural feature. While the obstruction 2 fully occludes the target object from line-of-sight perception relative to the control system 10 in this example, in embodiments, the target object may only be partially occluded in that line-of-sight or from other angles of perception by the obstruction.”
[0073] “The trajectory calculator 425 receives the RFID location information from the RFID reader 403 and the one or more extracted occluded regions identified and isolated by the extractor. After it has been determined that the target object is occluded, one or more predetermined models may be used to determine whether there is a trajectory calculated by the trajectory calculator 425 that may be taken by the robot to access the target object. …”
, where “identify[ing]” the “target object” from amongst one or more other objects (e.g., an “another object” or “obstruction”) is performing a target object recognition operation (e.g., recognizing an object as a “target object”) based on at least the extracted target region (e.g., “occluded regions identified and isolated by the extractor”)) and omits, based on the positioning information, a region of the original image outside of the target region and the one or more non-target objects in the region outside of the target region during the target object recognition operation (para(s). [0123-0124] and [0128]—see citation in claim limitation “extract a target region…” above—, where “eliminat[ing] areas of the image that are not proximate to the vicinity of the tag including the target object” is omitting a region of the original image outside of the target region (e.g., areas outside of the “cropp[ed]” image region); wherein the omitted region of the original image outside of the target region includes at least one or more non-target objects outside of the target region (e.g., non-target objects outside of the cropped “area 563” as depicted in Fig. 5 below:
PNG
media_image1.png
472
417
media_image1.png
Greyscale
)).
Regarding claim 5, Boroushaki discloses the recognition system according to claim 1, wherein Boroushaki further discloses the extract the target region includes
determining, based on the positioning information, a position and a size of the target region in the original image (para(s). [0123-0124] and [0128]—see citation in claim 1 limitation “extract a target region…”above—, where determin[ing] the “area around the location of the target object” including the “size” of the area is determining a position (e.g., “location”) and size of the target region (e.g., “area around… the target object”) in the original image), and
extracting the target region based on the position and the size of the target region (para(s). [0123-0124] and [0128]—see citation in claim 1 limitation “extract a target region…”above—, where “cropping” is extracting).
Regarding claim 8, Boroushaki discloses the recognition system of claim 1, wherein Boroushaki further discloses the acquire positioning information includes communicating with the target object using radio identification technology (para(s). [0037-0039] and [0057], recite(s)
[0037] “The control system 10 includes one or more tag readers 20 (or more simply “readers”), a controller 30, and at least one storage system 40 (e.g., a memory or other storage device). The tag reader(s) may receive signals from one or more tags 3 coupled to the occluded target object 1 (e.g. attached, disposed on or otherwise directly or indirectly coupled to or otherwise associated with or in the area of the occluded target object 1).”
[0038] “In embodiments, one or more tags may be provided as radio frequency identification (RFID) tags in which case one or more tag readers 20 may be provided as radio frequency identification (RFID) tag readers 20. For illustrative purposes, one RFID tag 3 is shown coupled or otherwise attached to the target object 1, but in embodiments, more than one RFID tag may be attached or otherwise coupled to the target object.”
[0039] “It should be understood that in embodiments, the tags may be different from RFID tags. For example, the target object may emit or otherwise produce Bluetooth signals, WiFi signals or other types of signals (that is, target objects may be said to be Bluetooth-tagged, WiFi-tagged, or tagged using another form of communication signal or standard or another form of wireless technology). To promote clarity in the description of the concepts sought to be protected by this patent, a tag may sometimes be referred to herein as an RFID tag. It should, however, be understood that the term “RF-tagged target object” (or more simply “tagged target object”) may include all forms of tags and communication signals and communication standards and technologies, not just RFID per se.”
[0057] “At 340, in response to the confirmation signal, the controller 30 may determine the location of the target object. In one embodiment, the location of the target object may be determined by mapping the position of the RFID signal relative to a predetermined reference point or frame of reference such as a coordinate system. …”
, where the “location of the target object may be determined by mapping the position of the RFID signal” is acquiring positioning information by at least communicating with the target object using radio identification technology (e.g., “RFID”)).
Regarding claim 9, Boroushaki discloses the recognition system of claim 8, wherein Boroushaki further discloses the radio identification technology includes at least one of radio-frequency identification (RFID), Bluetooth technology, ultra-wideband (UWB) technology, or wireless fidelity (WiFi) technology (para(s). [0037-0039]—see citations in claim 8 above—, where the radio identification technology includes at least “RFID”).
Regarding claim 10, Boroushaki discloses the recognition system of claim 1, wherein Boroushaki further discloses
the acquire the original image includes using an image capture device (para(s). [0032], [0036], and [0063]—see citations in claim 1 limitation “acquire an original image…” above—, where the “image sensor” is an image capture device), and
the acquire positioning information includes acquiring information regarding at least one of a distance, an elevation, or an azimuth angle of the target object in relation the image capture device (para(s). [0044] and [0063], recite(s)
[0044] “At 340, in response to the confirmation signal, the controller 30 may determine the location of the target object. In one embodiment, the location of the target object may be determined by mapping the position of the RFID signal relative to a predetermined reference point or frame of reference such as a coordinate system. In one embodiment, the RF-based location of the target object may be calculated as a three-dimensional (3D) location within a coordinate system having a predetermined origin (e.g., on the robot or at another location). The three-dimensional location may indicate the distance to and direction of the target object relative to the origin. The location (e.g., distance and direction) may be expressed radially or using a different method in other embodiments.”
[0063] “In one example implementation (described in greater detail below), a UR5e robot may be controlled based on visual information from a camera sensor, along with a customized RF localization system operating with one or more software radios for receiving an RFID signal from a tag. …”
, where determining the “location” of the target object from its transmitted “RFID signal” relative to a “predetermined reference point or frame [or origin]” (e.g., a “camera sensor, along with a customized RF localization system… for receiving an RFID signal from a tag”) is acquiring information regarding at least one of a distance (e.g., “location” comprises of “distance and direction”) of the target object in relation (e.g., “relative to”) the image capture device (e.g., “camera sensor”)).
Regarding claim 11, the claim is the method performed by the system of claim 1. Therefore, claim 11 recites similar limitations to claim 1 and is rejected for similar rationale and reasoning (see the analysis for claim 1 above).
Regarding claim 15, the claim recites similar limitations to claim 5 and is rejected for similar rationale and reasoning (see the analysis for claim 5 above).
Regarding claim 18, Boroushaki discloses a non-transitory computer readable storage medium, wherein the computer readable storage medium stores computer program instructions thereon, the computer program instructions, when executed by a processor, cause the processor to implement the method of claim 11 (para(s). [0044]—see citation in claim 1 limitation “processing circuitry” above—, where para(s). [0162] further recite(s):
[0162] “Also, another embodiment may include a computer-readable medium, e.g., a non-transitory computer-readable medium, for storing the code or instructions described above. The computer-readable medium may be a volatile or non-volatile memory or other storage device, which may be removably or fixedly coupled to the computer, processor, controller, or other signal processing device which is to execute the code or instructions for performing the method embodiments or operations of the apparatus embodiments described herein.”
).
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 2-4 and 12-14 are rejected under 35 U.S.C. 103 as being unpatentable over Boroushaki as applied to claims 1 and 11 above, and further in view of Nogami et al. (Nogami; US 2006/0022814 A1, cited in previously mailed Office Actions).
Regarding claim 2, Boroushaki discloses the recognition system according to claim 1, wherein the processing circuitry is further configured to
acquire information about the target object (para(s). [0054]—see citation in claim 1 limitation “acquire positioning information…” above—, wherein para(s). [0040], recite(s)
[0040] “When a declutter operation is performed, then a vision sensor may be able to distinguish (e.g., by silhouette or other feature recognition) between the untagged target object and the tagged object for purposes of accessing the target object. In one embodiment, the tag may be attached to an outside surface of the object. In another embodiment, the tag may be included in (i.e. beneath a surface of) or with the object, for example, by or as received by the manufacturer or other object provider.”
, where the “location” and/or “feature[s]” (e.g., “silhouette”) of a “target object” is/are information about the target object), and
wherein the recognizing of the target object includes
identifying feature information of the target object (para(s). [0040]—see preceding citation above—, where distinguishing “target object[s]” by “feature recognition” (e.g., “silhouette[s]”) is recognizing the target object by at least identifying feature information of the target object)
Where Boroushaki does not specifically disclose
identifying feature information of the target object based on the… target region; and
recognizing the target object by comparing the information about the target object with the feature information;
Nogami teaches in the same field of endeavor of target object recognition based on positioning information
identifying feature information of the target object based on the… target region (para(s). [0041], recite(s)
[0041] “A characteristic quantity comparator 105 acquires, from the database 104, only information corresponding to ID data of an RFID tag 20 read by the RFID reader 102. The acquired information contains characteristic quantity information of an object to which the RFID tag is attached, and information (e.g., the name) unique to the object. The characteristic quantity comparator 105 compares the characteristic quantities of an object acquired from the database 104 with the characteristic quantities of a sensed image acquired by the image processor 103, and determines the position of the object in the sensed image. The characteristic quantity comparison sequence is as follows. That is, the characteristic quantities, such as the edge, the color, the luminance, the texture, the presence/absence of a motion, the shape, and the distance between characteristic points, unique to the object acquired from the database 104 are compared with the characteristic quantities, such as the edge, the color, the luminance, the texture, the presence/absence of a motion, the shape, and the distance between characteristic points, of the sensed image obtained by the image processor 103, and a most matching portion is checked. If a matching portion is found, the ID data of the object and the position in the image are stored so that they correspond to each other. If a plurality of ID data is read by the RFID reader 102, the above sequence is repeated the same number of times as the number of the read ID data. As a consequence, a plurality of objects to which RFID tags are attached can be discriminated in the image. Note that if a plurality of ID data is read, information may also be simultaneously acquired from the database 104.”
, where the “acquired information contain[ing] characteristic quantity information” is identified feature information of the target object (e.g., “sensed object”) based on a target region (e.g., “position of the object in the sensed image”)); and
recognizing the target object by comparing the information about the target object with the feature information (para(s). [0041]—see preceding citation above—, where identifying the object in the “sensed image” by comparing or matching “characteristic quantities” acquired from the wireless network unite (e.g., the “RFID”) is recognizing the target object by comparing the information about the target object (e.g., the “acquired information contain[ing] characteristic quantity information”) with the feature information (e.g., the “characteristic quantities… of the sensed image”)).
Since Boroushaki and Nogami each disclose identifying a target object based on at least feature information and using a wireless network unit (e.g., RFID tag), it would have been obvious to one of ordinary skill in the art before the effective filing date of the presently filed invention to modify the system of Boroushaki to incorporate identifying the feature information of the target object based on the extracted target region and recognizing the target object by comparing the information about the target object with the feature information of the target object based on the extracted target region to improve target object recognition by increasing the confidence that the recognized target object is the desired target object by using feature comparison to distinguish the target object from non-target object(s) obstructing the target object and/or surrounding other non-target object(s) as taught by Nogami (para(s). [0054-0055], recite(s)
[0054] “In an actual operation, an object rarely independently exists in an image without being obstructed by anything. However, it is possible to detect the position of even an object slightly obstructed by another object or by a surrounding irrelevant article. That is, if even a portion of an object exists in a sensed image, the characteristic quantities of the portion are compared with those of an object obtained from the database 104. If the characteristic quantities are partially similar, it is determined that the object exits in this portion. In this manner, even when an RFID tag itself is completely invisible from the reader, the position of the object can be determined. That is, in the conventional technique which determines a position by using light, an RFID tag must be located in a position visible from the reader. In the first embodiment, however, it is possible to read an RFID tag intentionally located in a hidden position, or to read an RFID tag in a direction other than a surface to which the tag is attached.”
[0055] “FIGS. 7, 8, and 9 are views showing the way an object partially obstructed by another object is detected. FIG. 7 is a view showing the state in which objects shot by the image sensing unit 101 are displayed on the display unit 106. The RFID reader 102 reads RFID tags (not shown) attached to two objects 210 and 220. The RFID tag of the object 210 has ID data "00100", and the RFID tag of the object 220 has ID data "00101". The characteristic quantities of objects corresponding to these ID data and data (name data) unique to these objects are read out from the database 104 (FIG. 9). The characteristic quantities of the object 210 are "black, circular column". Therefore, the position of the object in the image can be simply determined by comparison with characteristic quantities obtained by processing the shot image.”
).
Regarding claim 3, Boroushaki discloses the recognition system according to claim 1, wherein Nogami teaches in the same field of endeavor of target object recognition based on positioning information the recognizing of the target object includes
identifying feature information of the target object based on the extracted target region (para(s). [0041]—see similar limitation in claim 2 above—, where the “acquired information contain[ing] characteristic quantity information” is identified feature information of the target object (e.g., “sensed object”)), and
recognizing the target object by comparing the feature information with a feature of the target region extracted based on the positioning information (para(s). [0041]—see similar limitation in claim 2 above—, where identifying the object in the “sensed image” by comparing or matching “characteristic quantities” is recognizing the target object by comparing the feature information (e.g., the “acquired information contain[ing] characteristic quantity information”) with a feature of the target region (e.g., “characteristic quantities… of the sensed image”)).
Since Boroushaki and Nogami each disclose identifying a target object based on at least feature information and using a wireless network unit (e.g., RFID tag), it would have been obvious to one of ordinary skill in the art before the effective filing date of the presently filed invention to modify the system of Boroushaki to incorporate identifying feature information of the target object based on the extracted target region and recognizing the target object by comparing the feature information with a feature of the target region extracted based on the positioning information to improve target object recognition by increasing the confidence that the recognized target object is the desired target object by using feature comparison to distinguish the target object from non-target object(s) obstructing the target object and/or surrounding other non-target object(s) as taught by Nogami (para(s). [0054-0055]—see citations in similar motivation to combine paragraph of claim 2 above).
Regarding claim 4, Boroushaki in view of Nogami discloses the recognition system according to claim 3, wherein Nogami further teaches
the feature information comprises at least one of a size and a location of the target object in the original image (para(s). [0066], [0083], and [0105], recite(s)
[0066] “FIGS. 11A and 11B illustrate cases in each of which an RFID tag 20 is read while the distance between an object 310 to which the RFID tag 20 is attached and the object information acquisition system 300 is changed. As shown in FIG. 11A, if radio field intensity 340 from the RFID tag 20 is low, the object 310 is far from the RFID reader 102. Therefore, if the image sensing unit 101 is installed adjacent to the RFID reader 102, the object 310 is far from the image sensing unit 101 as well. That is, the object is presumably sensed in a relatively small size in an image, so the characteristic quantity of the object 310 obtained from the image is also small. Accordingly, the reduction ratio of the characteristic quantity of the object is obtained from the radio field intensity and correspondence table, a characteristic quantity 312 of the object 310 read out from the database 104 is reduced to obtain a characteristic quantity 313, and the characteristic quantity 313 is compared with the characteristic quantity of an object 311 sensed in a small size. This facilitates the comparison. On the other hand, if the radio field intensity from the RFID tag 20 is high, as shown in FIG. 11B, the object is sensed in a large size in the sensed image. Therefore, the characteristic quantity 312 of the object 310 read out from the database 104 is enlarged to form a characteristic quantity 314, and the characteristic quantity 314 is compared with the characteristic quantity of the object 311 sensed in a large size.”
[0083] “…its object to match a sensing range with an RFID read range by controlling sensing parameters and RFID read parameters by synchronizing them with each other, when an RFID tag attached to a certain object is to be read while the object is being shot.”
[0105] “In the above arrangement, the parameter controller 530 controls the image sensing unit 510 and RFID reader 520 by synchronizing them with each other, so that a sensing range 11 in which the image sensing unit 510 senses an image and an RFID read range 12 in which the RFID reader 520 reads an RFID tag are substantially the same. In this way, only one object B can be contained in the sensing range 11 and RFID read range 12.”
, where comparing the characteristic quantities including “shape” (see para. [0041] in claim 3 above) include identifying the “size” of the target object in the database of “characteristic quantit[ies]” and determining the position of the target object in the image includes identifying the area where the target object is located based on the wireless network unit (e.g., a “RFID read range”) is the feature information further comprising at least one of a size and a location of the target object, respectively),
the feature of the extracted target region comprises at least one of a size and a location of the extracted target region (para(s). [0066], [0083], and [0105]—see preceding citation above—, where the “size” of the target object in the image and the image “sensing range” of the target object in the image are the features of the target object comprising at least one of a size and a location of the extracted target region, respectively (i.e., the size and location of the target object is the same as the size and location of the extracted target region as Krishnaswamy discloses the extracted target region is a region corresponding to the dimensions of the target object—see claim 1 limitation “extract a target region…” above—)), and
the recognizing of the target object includes comparing at least one of the size and the position of the target object in the original image with the corresponding at least one of the size and the position of the extracted target region (para(s). [0066], [0083], and [0105]—see preceding citations above—, where comparing the characteristic quantities including “shape” (see para. [0041] in claim 3 above) include identifying the “size” of the target object in the database of “characteristic quantit[ies]” and matching the position of the target object in the image with the position of the target object identified based on the wireless network unit is recognizing the target object by comparing at least one of the size and the position of the target object in the original image (e.g., the “size” and “sensing range” of the target object in the image, respectively) with the corresponding at least one of the size and the position of the extracted target region (e.g., the “size” and the “sensing range” based on the wireless network unit—the RFID—, respectively)).
Regarding claim 12, the claim recites similar limitations to claim 2 and is rejected for similar rationale and reasoning (see the analysis for claim 2 above).
Regarding claim 13, the claim recites similar limitations to claim 3 and is rejected for similar rationale and reasoning (see the analysis for claim 3 above).
Regarding claim 14, the claim recites similar limitations to claim 4 and is rejected for similar rationale and reasoning (see the analysis for claim 4 above).
Claims 6 and 16 are rejected under 35 U.S.C. 103 as being unpatentable over Boroushaki as applied to claims 1 and 11 above, and further in view of Krishnaswamy et al. (Krishnaswamy; US 2018/0276841 A1, cited in previously mailed Office Actions).
Regarding claim 6, Boroushaki discloses the recognition system according to claim 1, wherein Boroushaki further discloses the processing circuitry is further configured to
determine whether there is a plurality of target objects in the original image (para(s). [0157], recite(s)
[0157] “In some embodiments, multiple target objects may be located in the area of interest, which, for example, may or may not be cluttered or unstructured with no a priori knowledge by the control system of the area layout or contents. Thus, the control system may be designed to operate in any context where locating an occluded object is to be performed. In some embodiments, the control system may locate and/or retrieve target objects in multiple piles.”
, where locating “multiple target objects” is determining whether there is a plurality of target objects in the original image);
extract a plurality of target regions in the original image based on a plurality of positioning information of the plurality of target objects respectively based on a determination that there is the plurality of target objects, the plurality of target regions including at least one corresponding target object of the plurality of target objects (Since para(s). [0123-0124] and [0128]—see citations in “extract a target region…” above—discloses that for a single target object, a target region (e.g., “area around the location of the target object”) is extracted (e.g., “cropped”) for the target object based on at least positioning information (e.g., “location”) of the target object, a person of ordinary skill in the art before the effective filing date of the claimed invention would have recognized that in the embodiment comprising a plurality of target objects disclosed in para(s). [0157]—see citation in the preceding citation immediately above—, the process of extracting a target region for a single target object can be applied to the plurality of target objects to extract a plurality of target objects in the same manner),
recognize at least one of the plurality of target objects based on the target regions for recognition (para(s). [0043] and [0073]—see citation in claim 1 limitation “perform a target object recognition operation…” above—, where “identify[ing]” the “target object” from amongst one or more other objects (e.g., an “another object” or “obstruction”) for a plurality of target objects (as disclosed by para(s). [0157] above) is performing a target object recognition operation (e.g., “target object” identification) based on the target regions (e.g., “occluded regions identified and isolated by the extractor”)).
Where Boroushaki does not specifically disclose
merge adjacent target regions, of the plurality of target regions, into a merged target region, and
use the merged target region and a remainder of the plurality of target regions as target regions for recognition,
Krishnaswamy teaches in the same field of endeavor of extracting a plurality of regions of interest of a plurality of objects of interest
merge adjacent …regions, of the plurality of …regions, into a merged …region (para(s). [0083], recite(s)
[0033] “The use of the wireless network also provides the opportunity to increase efficiency in other ways. The use of the angle of transmission between the local device and individual objects permits a system to determine the position and identification of each of these multiple objects in an image (or scene or FOV). …The more objects are positioned and identified, the more efficient is the system with less conventional segmentation. Since the segmentation provides the regions of interest (ROIs) for the object detection, this also increases the efficiency of the object detection.”
[0083] “Process 800 may include “determine similarities between regions and merge regions to form final regions” 814. For this operation, numerous small regions may be merged to establish large similar regions that should be part of the same object. This may include a second pass of the same algorithms used to form the first rough regions, and/or connected component analysis techniques.”
, where “merg[ing] regions to form final regions” to establish a “same object” is merging adjacent regions of interest (e.g., regions desired for “object detection”) from a plurality of regions of interest into a merged region of interest), and
use the merged …region and a remainder of the plurality of …regions as target regions for recognition (para(s). [0092]—see citation above—, where providing the “objects or ROIS… for object detection” is using the merged region of interest (e.g., one of the “final regions” or refined regions) and a remainder of the plurality of regions of interest (e.g., regions not merged) as target regions for recognition (e.g., regions for “object detection” or classification)).
Since Boroushaki discloses extracting a plurality of target regions for a plurality of target objects, it would have been obvious to one of ordinary skill in the art before the effective filing date of the presently filed invention to try the method of Krishnaswamy comprising of merging adjacent regions of interest of an initial plurality of regions of interest into a merged region of interest and using the merged region of interest and a remainder of the plurality of regions of interest as target regions for recognition to the plurality of target regions of Boroushaki, such that the ‘target regions’ of Boroushaki are the ‘regions of interest’ in Krishnaswamy, in order to improve the extraction of the plurality of target regions for recognition when the extracted target regions for a plurality of target objects comprise and/or overlap with the same target object as taught by Krishnaswamy above.
Regarding claim 16, the claim recites similar limitations to claim 6 and is rejected for similar rationale and reasoning (see the analysis for claim 6 above).
Claims 7 and 17 are rejected under 35 U.S.C. 103 as being unpatentable over Boroushaki in view of Krishnaswamy as applied to claims 6 and 16 above, and further in view of Nogami et al. (Nogami; US 2006/0022814 A1, cited in previously mailed Office Actions).
Regarding claim 7, Boroushaki in view of Krishnaswamy, discloses the recognition system according to claim 6, wherein Nogami teaches in the same field of endeavor of target object recognition based on positioning information the recognizing of the at least one of the plurality of target objects includes
identifying a plurality of feature information of the plurality of target objects based on the target regions for recognition (para(s). [0041]—see claim 2 limitation “identifying feature information…” above—, where para(s). [0112] further recite(s):
[0112] “…Note that if a plurality of objects is sensed, it is also possible to perform sensing again.”
, where the identified feature information (e.g., “acquired information contain[ing] characteristic quantity information”) of a target object (e.g., “sensed object”) can be performed for a plurality of target objects (e.g., “plurality of [sensed] objects”) is identifying a plurality of feature information of the plurality of target objects), and
recognizing the at least one of the plurality of target objects by comparing the plurality of feature information with features of the plurality of target regions extracted based on the plurality of positioning information (para(s). [0041]—see claim 2 limitation “recognizing the target object by…” above—, where para(s). [0055] further recite(s):
[0055] “FIGS. 7, 8, and 9 are views showing the way an object partially obstructed by another object is detected. FIG. 7 is a view showing the state in which objects shot by the image sensing unit 101 are displayed on the display unit 106. The RFID reader 102 reads RFID tags (not shown) attached to two objects 210 and 220. The RFID tag of the object 210 has ID data "00100", and the RFID tag of the object 220 has ID data "00101". The characteristic quantities of objects corresponding to these ID data and data (name data) unique to these objects are read out from the database 104 (FIG. 9). The characteristic quantities of the object 210 are "black, circular column". Therefore, the position of the object in the image can be simply determined by comparison with characteristic quantities obtained by processing the shot image.”
, where “discrminat[ing]” a “plurality of objects to which RFID tags are attached” is recognizing at least one of the plurality of target objects by comparing the plurality of feature information (e.g., the “acquired information contain[ing] characteristic quantity information”) with features of the plurality of target regions (e.g., the “characteristic quantity information” of the objects sensed in the image)).
Since Boroushaki and Nogami each disclose identifying a target object based on at least feature information and using a wireless network unit (e.g., RFID tag), it would have been obvious to one of ordinary skill in the art before the effective filing date of the presently filed invention to modify the system of Boroushaki to incorporate identifying a plurality of feature information of the plurality of target objects based on the extracted target regions and recognizing at least one of the plurality of target objects by comparing the plurality of feature information about the target objects with features of the plurality of target objects based on the extracted target region to improve target object recognition by increasing the confidence that the recognized target object is the desired target object by using feature comparison to distinguish the target object from non-target object(s) obstructing the target object and/or surrounding other non-target object(s) as taught by Nogami (para(s). [0054-0055]—see citations in similar motivation to combine paragraph of claim 2 above).
Regarding claim 17, the claim recites similar limitations to claim 7 and is rejected for similar rationale and reasoning (see the analysis for claim 7 above).
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JULIA Z YAO whose telephone number is (571)272-2870. The examiner can normally be reached Monday - Friday (8:30AM - 5PM).
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Emily Terrell can be reached at (571)270-3717. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/J.Z.Y./Examiner, Art Unit 2666
/MING Y HON/Primary Examiner, Art Unit 2666
Prosecution Insights