Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
Applicant’s arguments with respect to claim(s) 1-8, 10-15 have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
With regards to the Cho reference discussed in the Remarks, pages 7-8, newly found prior art is relied on to teach the limitations of claim 1. Cho is only relied on to teach the limitations of dependent claim 10 regarding outline color processing.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claim(s) 11-12 is/are rejected under 35 U.S.C. 102(a)(1) as being anticipated by WO 2019196133 to Li.
Regarding claim 11, Li discloses an image processing method performed by an image processing device (paragraph 50; method for head mounted device includes image processing module 300), the image processing method comprising:
obtaining, from a first image, object information about an object that attracts a user's attention included in the first image for image processing (paragraph 86, 94-95; object information such as location is obtained from the captured image (first image) for contour enhancement (image processing); paragraph 95, 108-109; eye-tracking module 600 identifies target area for objects that user focuses on (attracts attention) for image enhancement);
receiving a user control command for image processing, wherein the user control command specifies at least one of outline processing or flattening processing to be applied to the object that attracts the user's attention (paragraph 52-53, 55, 95; receiving user input command via input module 200 for outline enhancement (image processing) of the target object that user is focused on via eye-tracking as attracting attention; user input command specifies outline processing);
obtaining control information including the user control command for image processing (paragraph 52-53, 55; image processing module 300 obtains the user control information that includes the user input command that controls the image processing via input module 200); and
obtaining, from the first image, a second image by performing image processing on the object that attracts the user's attention based on the object information and user control information (paragraph 52-54, 55, 108; the “image after image enhancement” (second image) (see paragraph 54) is obtained from the image processing module 300 that performs image processing on first image to enhance the outline of the object that user focuses on (attention) using the position information (object information) of the target area and user input command (control information)),
wherein the at least one of the outline processing or the flattening processing is performed based on a parameter from the user for the object (paragraph 69, 80; the outline processing is performed based on matrix values (parameter) of Sobel operator that is adjusted by user for the object enhancement).
Regarding claim 12, Li discloses the image processing method of claim 11, wherein the object information comprises information about at least one of a type, location, and size of the object that attracts the user's attention (paragraph 86, 94-95; object information such as location is obtained from the captured image for contour enhancement (image processing); paragraph 95, 108-109; eye-tracking module 600 identifies target area for objects that user focuses on (attracts attention) for image enhancement).
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1, 2, 4, 5, 14, 15 is/are rejected under 35 U.S.C. 103 as being unpatentable over WO 2019196133 to Li in view of US 20170147174 to OLEJNICZAK.
Regarding claim 1, Li discloses an image processing device comprising (paragraph 50; head mounted device includes image processing module 300):
obtain, from a first image, object information about an object that attracts a user's attention for image processing (paragraph 86, 94-95; object information such as location is obtained from the captured image for contour enhancement (image processing); paragraph 95, 108-109; eye-tracking module 600 identifies target area for objects that user focuses on (attracts attention) for image enhancement),
receive a user control command for image processing, wherein the user control command specifies at least one of outline processing or flattening processing to be applied to the object that attracts a user's attention (paragraph 52-53, 55, 95; receiving user input command via input module 200 for outline enhancement (image processing) of the target object that user is focused on via eye-tracking as attracting attention; user input command specifies outline processing),
obtain control information including the user control command (paragraph 52-53, 55; image processing module 300 obtains the user control information that includes the user input command that controls the image processing via input module 200), and
obtain a second image by performing the image processing on the object that attracts the user's attention based on the object information and the control information (paragraph 52-54, 55, 108; the “image after image enhancement” (second image) (see paragraph 54) is obtained from the image processing module 300 that performs image processing to enhance the outline of the object that user focuses on (attention) using the position information (object information) of the target area and user input command (control information)),
wherein the at least one of the outline processing or the flattening processing is performed based on a parameter from the user for the object (paragraph 69, 80; the outline processing is performed based on matrix values (parameter) of Sobel operator that is adjusted by user for the object enhancement).
However Li does not disclose image processing device comprising a memory storing one or more instructions; and a processor configured to execute the one or more instructions stored in the memory.
OLEJNICZAK discloses image processing device comprising a memory storing one or more instructions; and a processor configured to execute the one or more instructions stored in the memory (paragraph 84, 259; program stored in storage module 191 is executed by processor).
It would have been obvious to one of ordinary skill in the art at the time of the invention was made to modify the system of Li as taught by OLEJNICZAK to provide programming processor for image processing.
The motivation to combine the references is to allow user control to control area where the image processing is performed and allow user to move the generated second image and further provide processor that can be programmed to perform the method (paragraph 72).
Regarding claim 2, Li discloses the image processing device of claim 1, wherein the object information includes information about at least one of a type, location, and size of the object that attracts the user's attention (paragraph 86, 94-95; object information such as location is obtained from the captured image for contour enhancement (image processing); paragraph 95, 108-109; eye-tracking module 600 identifies target area for objects that user focuses on (attracts attention) for image enhancement).
Regarding claim 4, OLEJNICZAK discloses the image processing device of claim 1, wherein the user control command further specifies at least one of whether to enlarge an object or a degree of enlargement of the object (paragraph 166-167; based on user command/input, the zoom box is displayed to enlarge objects in the box).
Regarding claim 5, Li discloses the image processing device of claim 1, wherein the processor is further configured to execute the one or more instructions to perform, according to the control information, at least one of upscaling of the object, the outline processing around the object, and the flattening processing of an inside of the object (paragraph 52-53, 55, 95; receiving user input command (control information) via input module 200 for outline/contour enhancement (image processing) of the target object ; paragraph 81; enhancing around “defect area”).
Regarding claim 14, OLEJNICZAK discloses the image processing method of claim 11, wherein the user control command further specifies at least one of whether to enlarge an object or a degree of enlargement of the object (paragraph 166-167; based on user command/input, the zoom box is displayed to enlarge objects in the box).
Regarding claim 15, Li discloses the image processing method comprising (paragraph 50; head mounted device includes method for image processing module 300):
obtaining, from a first image, object information about an object that attracts a user's attention included in the first image for image processing (paragraph 86, 94-95; object information such as location is obtained from the captured image for contour enhancement (image processing); paragraph 95, 108-109; eye-tracking module 600 identifies target area for objects that user focuses on (attracts attention) for image enhancement);
receiving a user control command for image processing, wherein the user control command specifies at least one of outlines processing or flattening processing to be applied to the object that attracts the user's attention (paragraph 52-53, 55, 95; receiving user input command via input module 200 for outline enhancement (image processing) of the target object that user is focused on via eye-tracking as attracting attention; user input command specifies outline processing);
obtaining control information including the user control command for image processing (paragraph 52-53, 55; image processing module 300 obtains the user control information that includes the user input command that controls the image processing via input module 200); and
obtaining, from the first image, a second image by performing image processing on the object that attracts the user's attention based on the object information and user control information (paragraph 52-54, 55, 108; the “image after image enhancement” (second image) (see paragraph 54) is obtained from the image processing module 300 that performs image processing on first image to enhance the outline of the object that user focuses on (attention) using the position information (object information) of the target area and user input command (control information)),
wherein the at least one of the outline processing or the flattening processing is performed based on a parameter from the user for the object (paragraph 69, 80; the outline processing is performed based on matrix values (parameter) of Sobel operator that is adjusted by user for the object enhancement).
However Li does not disclose a non-transitory computer-readable recording medium having recorded thereon a program for implementing an image processing method.
OLEJNICZAK discloses a non-transitory computer-readable recording medium having recorded thereon a program for implementing an image processing method (paragraph 84, 259; program stored in storage module 191 is executed by processor for implementing image processing).
It would have been obvious to one of ordinary skill in the art at the time of the invention was made to modify the system of Li as taught by OLEJNICZAK to provide programming processor for image processing.
The motivation to combine the references is to allow user control to control area where the image processing is performed and allow user to move the generated second image and further provide processor that can be programmed to perform the method (paragraph 72).
Claim(s) 3 is/are rejected under 35 U.S.C. 103 as being unpatentable over WO 2019196133 to Li in view of US 20170147174 to OLEJNICZAK further in view of US 20110170742 to Fukuchi.
Regarding claim 3, Li discloses the object that attracts the user's attention (paragraph 95, 108-109; eye-tracking module 600 identifies target area for objects that user focuses on (attracts attention) for image enhancement). However Li does not disclose the image processing device of claim 1, wherein the processor is further configured to execute the one or more instructions to:
detect a plurality of objects from the first image,
output object identification information indicating the plurality of objects, and
identify an object selected by the user to correspond to the outputting of the object identification information.
Fukuchi discloses wherein the processor is further configured to execute the one or more instructions to (paragraph 139; CPU executing program):
detect a plurality of objects from the first image (paragraph 123; plurality of objects detected in image),
output object identification information indicating the plurality of objects (paragraph 105; object identifier output based on detection of objects), and
identify an object selected by the user to correspond to the outputting of the object identification information (paragraph 129, 136; user selects object to be operated (important object) is identified that corresponds with object ID outputted).
It would have been obvious to one of ordinary skill in the art at the time of the invention was made to modify the system of Li as taught by Fukuchi to provide object ID to plurality of objects in the image.
The motivation to combine the references is to associate object identification to the objects in the image by using object ID data in order to assist in selecting certain objects in the image for analysis (paragraph 147-148).
Claim(s) 6 is/are rejected under 35 U.S.C. 103 as being unpatentable over WO 2019196133 to Li in view of US 20170147174 to OLEJNICZAK further in view of US 20040228528 to Lao.
Regarding claim 6, OLEJNICZAK disclose the image processing device of claim 1, wherein the control information comprises real-time user control information, the real-time user control information comprises real-time control information of the user for the first image (paragraph 72, 152; user can control the region for enlarging (image processing) by letting user move the region via sensor (real time control) to be enlarged in the first image).
However Li in view of OLEJNICZAK does not disclose wherein the control information comprises at least one of inference control information, the inference control information is obtained from previous control history information of a user for a previous image.
Lao discloses wherein the control information comprises at least one of inference control information, the inference control information is obtained from previous control history information of a user for a previous image (paragraph 82-83, 114-115; user DB 47 stores previous user image correction parameters performed as history for user on previous face images; in step st24 based on image face features, the user DB 47 is searched for matching features and the associated correction parameters are inferred as inference control info for correcting the current image).
It would have been obvious to one of ordinary skill in the art at the time of the invention was made to modify the system of Li in view of OLEJNICZAK as taught by Lao to provide inference control based on history information.
The motivation to combine the references is to provide efficient image processing by reusing past image processing parameters for processing current image data that is similar to past images thereby maintaining image processing for similar images (paragraph 82-83, 114-115).
Claim(s) 7, 8 is/are rejected under 35 U.S.C. 103 as being unpatentable over WO 2019196133 to Li in view of US 20170147174 to OLEJNICZAK further in view of US 20220222872 to Ghosh.
Regarding claim 7, Li does not disclose the image processing device of claim 1, wherein the processor is further configured to execute the one or more instructions to obtain the second image from the first image by using a neural network, and the neural network is a neural network that has learned an input image, an object area of interest to a user in the input image, and a ground truth image obtained by processing the object area of interest to the user, as a learning data set.
Ghosh discloses wherein the processor is further configured to execute the one or more instructions to obtain the second image from the first image by using a neural network (paragraph 53; processor 505; paragraph 31; neural network 200 inputs image and generates output image (second image) ) , and the neural network is a neural network that has learned an input image (paragraph 9, 27;neural network models trained based on input image), an object area of interest to a user in the input image (paragraph 32, 43; user region of interest such as sky region of input image), and a ground truth image obtained by processing the object area of interest to the user, as a learning data set (paragraph 40; ground truth image (learning data set) used for learning having stylized (processed) for region of interest).
It would have been obvious to one of ordinary skill in the art at the time of the invention was made to modify the system of Li as taught by Ghosh to provide trained neural network to generate second image.
The motivation to combine the references is to provide neural network that generates second image in simple operation that requires users to provide a specific style and wherein the neural network automatically generates stylized second image from input image by performing processing based on selected style to important object regions in the image (paragraph 8-9).
Regarding claim 8, Ghosh discloses the image processing device of claim 7, wherein the neural network obtains the second image in which an object is image processed from at least one of the first image, the control information, and the object information (paragraph 9; stylization (image processing) applied to sky, face objects; paragraph 31; neural network 200 inputs image and generates output image (second image) ; paragraph 43; user selected style (step 406) is used as control information for generating second image).
Claim(s) 10 is/are rejected under 35 U.S.C. 103 as being unpatentable over WO 2019196133 to Li in view of US 20170147174 to OLEJNICZAK further in view of US 20220222872 to Ghosh further in view of US 20210092472 to Cho.
Regarding claim 10, Li discloses the image processing device of claim 8, wherein the image processing comprises at least one of outline processing of the object, flattening processing of an inside of the object, and upscaling of the object (paragraph 52-53, 55, 95; receiving user input command via input module 200 for outline enhancement (image processing) of the target object).
However Li does not disclose the outline processing of the object comprises processing of at least one of a detail, intensity, and color of an outline of the object, the flattening processing of the inside of the object comprises processing of adjusting a flattening degree of the inside of the object, and the upscaling of the object comprises processing of enlarging a size of the object while maintaining a resolution of the object.
Cho discloses wherein the image processing comprises at least one of outline processing of the object, flattening processing of an inside of the object, and upscaling of the object (paragraph 98-99; determining outline thickness of the object),
the outline processing of the object comprises processing of at least one of a detail, intensity, and color of an outline of the object (paragraph 143-145; outline of object has color is determined),
the flattening processing of the inside of the object comprises processing of adjusting a flattening degree of the inside of the object, and
the upscaling of the object comprises processing of enlarging a size of the object while maintaining a resolution of the object.
It would have been obvious to one of ordinary skill in the art at the time of the invention was made to modify the system of Li in view of OLEJNICZAK as taught by Cho to provide outline processing to first image objects.
The motivation to combine the references is to provide complementary colors for the outline of objects so that the objects can be differentiated from the background area of the image near the outline (paragraph 152-153).
4. Claim(s) 13 is/are rejected under 35 U.S.C. 103 as being unpatentable over WO 2019196133 to Li in view of US 20110170742 to Fukuchi.
Regarding claim 13, Li discloses the object that attracts the user's attention (paragraph 95, 108-109; eye-tracking module 600 identifies target area for objects that user focuses on (attracts attention) for image enhancement).
However Li does not disclose the image processing method of claim 11, further comprising:
detecting a plurality of objects from the first image; outputting object identification information indicating the plurality of objects; and identifying an object selected by a user as the object to correspond to the outputting of the object identification information.
Fukuchi discloses detecting a plurality of objects from the first image (paragraph 123; plurality of objects detected in image); outputting object identification information indicating the plurality of objects (paragraph 105; object identifier output based on detection of objects); and identifying an object selected by a user as the object to correspond to the outputting of the object identification information (paragraph 129, 136; user selects object to be operated (important object) is identified that corresponds with object ID outputted).
It would have been obvious to one of ordinary skill in the art at the time of the invention was made to modify the system of Li as taught by Fukuchi to provide object ID to plurality of objects in the image.
The motivation to combine the references is to associate object identification to the objects in the image by using object ID data in order to assist in selecting certain objects in the image for analysis (paragraph 147-148).
Allowable Subject Matter
Claim 9 is objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Other Prior Art Cited
14. The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
US-20070147700 to Jeong discloses image object contour processing (see abstract).
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to BENIYAM MENBERU whose telephone number is (571) 272-7465. The examiner can normally be reached on Monday-Friday, 10:00am-6:30pm.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Akwasi Sarpong can be reached on (571) 270-3438. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Any inquiry of a general nature or relating to the status of this application or proceeding should be directed to the customer service office whose telephone number is (571) 272-2600. The group receptionist number for TC 2600 is (571) 272-2600.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only.
For more information about the PAIR system, see <http://pair-direct.uspto.gov/>. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free).
Patent Examiner
Beniyam Menberu
/BENIYAM MENBERU/Primary Examiner, Art Unit 2681
05/02/2026
Prosecution Insights