DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Priority
Receipt is acknowledged of certified copies of papers required by 37 CFR 1.55. Certified copies of Korean Patent Application No. 10-2023-017-2023-01179969 and 10-2023-0114001 were electronically retrieved May 4, 2026.
Status of Claims
The Office Action is in response to the remarks and amendments filed on 5/26/2026. Accordingly, claims 1-20 are pending for consideration in this Office Action.
Claim Objections
Claim objected to because of the following informalities:
Regarding Claim 14, the recitation “state of being completely and halted during the specific time” should be - - state of being completely open and halted during the specific time - -.
Regarding Claim 19, the recitation “state of being completely and halted during the specific time” should be - - state of being completely open and halted during the specific time - -.
Appropriate correction is required.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 2, 9, 11, 16, 17 and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Zhao (CN105472337A) in view of Takeshi et al. (WO2022038799A1).
Regarding Claim 1, Zhao teaches a refrigerator [where an image of a specified designated is captured and where the designated space refers to the internal space of storage equipment such as refrigerators; 0058-0059] comprising:
a body comprising a storage chamber [where the refrigerator is a storage device with an internal space called a designated space; 0059];
a door [where the storage device includes a door; 0104] rotatably coupled to the body to open or close the storage chamber [where an operator opens the door of the storage device to take away an item; 0069; where in a specific implementation a sensor device can obtain the motion state, open or closed, of the door from rotation angular velocity and rotation angle; 0072-0074];
a camera on the body [where the camera device can be installed inside the storage device; 0071], the camera being configured to capture an image of an inside of the body [where the camera device can use an image sensor to capture surveillance images of the designated space; 0071];
a memory [a storage medium including USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, optical disks, and other media that can store program code; 0109] configured to store one or more instructions [where the software functional units stored in a storage medium include several instructions to cause a computer device or processor to execute some steps; 0109]; and
at least one processor [a processor; 0109] operatively coupled to the memory and configured to execute the one or more instructions stored in the memory [0109], wherein the one or more instructions, when executed by the at least one processor, cause the refrigerator to:
obtain a template image [where a second image of the specified space taken in the previous capture can be further acquired; 0077] based on the door being open [where if the rotational angular velocity is positive and greater than or equal to the preset opening threshold, the door can be determined to be open; 0074] and being halted [where after the door opens, if the acceleration is less than or equal to the preset stationary threshold, the door can be determined to be stationary; 0074] during a specific time [where one of ordinary skill in the art would understand velocity and acceleration are measured with respect to a duration of time, where when the door is stationary, a start command can be sent to the camera device; when the door is closed, a stop command can be sent to the camera device; 0075],
based on detecting opening of the door, control the camera to capture the image of at least a part of the inside of the body [where step 101 is to obtain the first image of the specified space captured in the current shot; 0058; where when the door is stationary, a start command can be sent to the camera device; when the door is closed, a stop command can be sent to the camera device; 0075],
identify, based on the template image [the second image; the image from the previous capture; 0077; 0086] and the image captured by the camera [a first image of a specified space currently captured; 0086], whether at least one of a first area of the inside of the body included in the image satisfies a predetermined condition [where if the number of pixel value is greater than or equal to a preset pixel threshold it is determined that the specified space has not changed and if the number of pixels with the same pixel value is less than the pixel threshold, it is determine the specified space has changed; 0082], and
based on identifying that the predetermined condition is satisfied, store the image as an optimal image [where if the specified space has changed the first image is uploaded to the server to be processed and to update the processing results to the monitoring platform; 0085; 0086].
Zhao does not teach where the door comprises a door bin where the camera is configured to capture an image of an inside of the door and where the one or more instructions cause the refrigerator to, based on detecting opening of the door control the camera to capture the image of the door and storing the image as an optimal door bin image.
However, Takeshi teaches a method of processing image data in a refrigerator [0001] where the door comprises a door bin [door storage section 11a and 11b, Figure 11;0038] where the camera [image pickup device 30, Figure 1;0030] is configured to capture an image of an inside of the door [where the image captured captures the storage compartment and the entire door storage section, Figure 11; 0035] where the one or more instructions [where control SoC 41 issues a shooting instruction to camera 33; 0023] cause the refrigerator to, based on detecting opening of the door [when SoC41 detects the door is opened; 0023], control the camera to capture the image of the door [0023] and store the image as an optimal door bin image [where after the SoC 41 performs image output processing on the processed image and the image is updated; 0029] where one of ordinary skill in the art would have been capable of applying this known technique, capturing storage with a wide-angle fisheye lens, to a known device, a refrigerator, that was ready for improvement and the results would have been predictable to one of ordinary skill in the art i.e., allowing the entire interior of the fridge to be viewed [Takeshi; 0018].
Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the invention to modify the assembly of Zhao to have where the camera is configured to capture an image of an inside of the door where the one or more instructions cause the refrigerator to, based on detecting opening of the door, control the camera to capture the image of the door and based on identifying that the predetermined condition is satisfied, store the image as an optimal door bin image in view of the teachings of Takeshi where this known technique could have been applied to a known device that was ready for improvement and the results would have been predictable i.e., allowing the entire interior of the fridge to be viewed [Takeshi; 0018].
Regarding Claim 2, Zhao, as modified, teaches the invention of claim 1 and does not teach where the camera is on an upper end of the body, wherein the first area of the inside of the body is an area on a lower end and a side surface of the body covered by the door in a state in which the door is closed, and wherein the first line is on an upper end of the door.
However, Takeshi teaches a method of processing image data in a refrigerator [0001] where the camera is on an upper end of the body [image pickup device 30, Figure 1;0030], wherein the first area [storage compartment 11c, Figure 11; 0038] of the inside of the body is an area on a lower end and a side surface of the body covered by the door in a state in which the door is closed [a side of refrigerator 100 with box body 10, Figure 1 and annotated Figure 11], and wherein the first line is on an upper end of the door [first line of door storage section 11b, annotated Figure 11, where a door inherently includes lines to form its geometry] where one of ordinary skill in the art would have been capable of applying this known technique, a wide-angle fisheye lens, to a known device that was ready for improvement and the results would have been predictable to one of ordinary skill in the art i.e., allowing the entire interior of the fridge to be viewed with a single camera [Takeshi; 0018].
Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the invention to modify the assembly of the combined teachings to have where the camera is on an upper end of the body, wherein the first area of the inside of the body is an area on a lower end and a side surface of the body covered by the door in a state in which the door is closed, and wherein the first line is on an upper end of the door in view of the teachings of Takeshi where this known technique could have been applied to a known device that was ready for improvement and the results would have been predictable i.e., allowing the entire interior of the fridge to be viewed with a single camera [Takeshi; 0018].
Regarding Claim 9, Zhao, as modified, teaches the invention of claim 1 and does not teach a display, wherein the one or more instructions, when executed by the at least one processor, further cause the refrigerator to: rotate the optimal door bin image to be aligned with a horizontal line based on a conversion guideline, obtain an image of an area of interest by extracting the area of interest in the rotated optimal door bin image and cropping the area of interest, correct the image of the area of interest by applying a perspective transformation, and based on receiving a user input to output the optimal door bin image, control the display to output the corrected image of the area of interest.
However, Takeshi teaches a method of processing image data in a refrigerator [0001] to rotate the optimal door bin image to be aligned [where microcomputer 52 performs an image rotation process to rotate the image of each storage section; 0041] with a horizontal line based on a conversion guideline [in the field of view from the front the viewing position of the user; 0041], obtain an image of an area of interest by extracting the area of interest in the rotated optimal door bin image [a food storage area; 0044] and cropping the area of interest [where the cropping process can remove at least a portion of the area that is not a food storage area from the input photographed image; 0044], correct the image of the area of interest by applying a perspective transformation [where microcomputer performs distortion correction in the depth direction to make it rectangular; 0042], and based on receiving a user input to output the optimal door bin image, control the display to output the corrected image of the area of interest [where server 54 transmits fridge interior image data in response to a request to display a fridge interior image from a mobile device; 0024] where one of ordinary skill in the art would have been capable of applying this known technique, processing the door bin image, to a known device that was ready for improvement and the results would have been predictable to one of ordinary skill in the art i.e., providing an image of the interior of the refrigerator that is close to the state observed by the user while minimizing image switching operations on the display operation device. [Takeshi; 0060].
Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the invention to modify the assembly of the combined teachings to have wherein the one or more instructions, when executed by the at least one processor, further cause the refrigerator to: rotate the optimal door bin image to be aligned with a horizontal line based on a conversion guideline, obtain an image of an area of interest by extracting the area of interest in the rotated optimal door bin image and cropping the area of interest, correct the image of the area of interest by applying a perspective transformation, and based on receiving a user input to output the optimal door bin image, control the display to output the corrected image of the area of interest in view of the teachings of Takeshi where this known technique could have been applied to a known device that was ready for improvement and the results would have been predictable i.e., providing an image of the interior of the refrigerator that is close to the state observed by the user while minimizing image switching operations on the display operation device. [Takeshi; 0060].
Regarding Claim 11, Zhao teaches a controlling method of a refrigerator [where an image of a specified designated is captured and where the designated space refers to the internal space of storage equipment such as refrigerators; 0058-0059] comprising
a body comprising a storage chamber [where the refrigerator is a storage device with an internal space called a designated space; 0059],
a door [where the storage device includes a door; 0104] and rotatably coupled to the body to open or close the storage chamber [where an operator opens the door of the storage device to take away an item; 0069; where in a specific implementation a sensor device can obtain the motion state, open or closed, of the door from rotation angular velocity and rotation angle; 0072-0074], and
a camera on the body [where the camera device can be installed inside the storage device; 0071], the controlling method comprising:
obtaining a template image [where a second image of the specified space taken in the previous capture can be further acquired; 0077] based on the door being open [where if the rotational angular velocity is positive and greater than or equal to the preset opening threshold, the door can be determined to be open; 0074] and being halted [where after the door opens, if the acceleration is less than or equal to the preset stationary threshold, the door can be determined to be stationary; 0074] during a specific time [where one of ordinary skill in the art would understand velocity and acceleration are measured with respect to a duration of time; where when the door is stationary, a start command can be sent to the camera device; when the door is closed, a stop command can be sent to the camera device; 0075];
based on detecting opening of the door, controlling the camera to capture an image of at least a part of an inside of the body [where step 101 is to obtain the first image of the specified space captured in the current shot; 0058; where when the door is stationary, a start command can be sent to the camera device; when the door is closed, a stop command can be sent to the camera device; 0075 ];
identifying, based on the template image [the second image; the image from the previous capture; 0077; 0086] and the image captured by the camera [a first image of a specified space currently captured; 0086], whether at least one of a first area of the inside of the body included in the image satisfies a predetermined condition [where if the number of pixel value is greater than or equal to a preset pixel threshold it is determined that the specified space has not changed and if the number of pixels with the same pixel value is less than the pixel threshold, it is determine the specified space has changed; 0082]; and
based on identifying that the predetermined condition is satisfied, storing the image as an optimal image [where if the specified space has changed the first image is uploaded to the server to be processed and to update the processing results to the monitoring platform; 0085; 0086].
Zhao does not teach the door comprising a door bin where the method comprises controlling the camera to capture an image of the door and storing the image as an optimal door bin image.
However, Takeshi teaches a method of processing image data in a refrigerator [0001] including the door comprising a door bin [door storage section 11a and 11b, Figure 11;0038 ] and controlling the camera [image pickup device 30, Figure 1;0030] to capture an image of the door [where the image captured captures the storage compartment and the entire door storage section, Figure 11; 0035] and storing the image as an optimal door bin image [where after the SoC 41 performs image output processing on the processed image and the image is updated; 0029; where the image captured captures the storage compartment and the entire door storage section, Figure 11; 0035] where one of ordinary skill in the art would have been capable of applying this known technique, capturing door storage with a wide-angle fisheye lens, to a known device, a refrigerator, that was ready for improvement and the results would have been predictable to one of ordinary skill in the art i.e., allowing the entire interior of the fridge to be viewed [Takeshi; 0018].
Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the invention to modify the assembly of Zhao to have where the door comprising a door bin where the method comprises controlling the camera to capture an image of the door and storing the image as an optimal door bin image in view of the teachings of Takeshi where this known technique could have been applied to a known device that was ready for improvement and the results would have been predictable i.e., allowing the entire interior of the fridge to be viewed [Takeshi; 0018].
Regarding Claim 12, Zhao, as modified, teaches the method of claim 11 and does not teach wherein the camera is located on an upper end of the body, wherein the first area of the inside of the body is an area on a lower end and a side surface of the body covered by the door in a state in which the door is closed, and wherein the first line is on an upper end of the door.
However, Takeshi teaches a method of processing image data in a refrigerator [0001] where the camera is on an upper end of the body [image pickup device 30, Figure 1;0030], wherein the first area [storage compartment 11c, Figure 11; 0038] of the inside of the body is an area on a lower end and a side surface of the body covered by the door in a state in which the door is closed [a side of refrigerator 100 with box body 10, Figure 1 and annotated Figure 11], and wherein the first line is on an upper end of the door [first line of door storage section 11b, annotated Figure 11, where a door inherently includes lines to form its geometry] where one of ordinary skill in the art would have been capable of applying this known technique, a wide-angle fisheye lens, to a known device that was ready for improvement and the results would have been predictable to one of ordinary skill in the art i.e., allowing the entire interior of the fridge to be viewed with a single camera [Takeshi; 0018].
Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the invention to modify the assembly of the combined teachings to have where the camera is on an upper end of the body, wherein the first area of the inside of the body is an area on a lower end and a side surface of the body covered by the door in a state in which the door is closed, and wherein the first line is on an upper end of the door in view of the teachings of Takeshi where this known technique could have been applied to a known device that was ready for improvement and the results would have been predictable i.e., allowing the entire interior of the fridge to be viewed with a single camera [Takeshi; 0018].
Regarding Claim 16, Zhao teaches a non-transitory computer readable medium [a computer readable storage medium which may be USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, optical disks, and other media that can store program code; 0109] having instructions stored therein [where the software functional units are stored in a storage medium and include several instructions to cause a computer device or processor to execute some steps; 0109] which when executed by a processor [a processor; 0109] of a refrigerator comprising a body comprising a storage chamber [where the refrigerator is a storage device with an internal space called a designated space; 0059], a door [where the storage device includes a door; 0104] and rotatably coupled to the body to open or close the storage chamber [where an operator opens the door of the storage device to take away an item; 0069; where in a specific implementation a sensor device can obtain the motion state, open or closed, of the door from rotation angular velocity and rotation angle; 0072-0074], and a camera located on the body [where the camera device can be installed inside the storage device; 0071], cause the processor to execute a method comprising: obtaining a template image [where a second image of the specified space taken in the previous capture can be further acquired; 0077] based on the door being open [where if the rotational angular velocity is positive and greater than or equal to the preset opening threshold, the door can be determined to be open; 0074] and being halted [where after the door opens, if the acceleration is less than or equal to the preset stationary threshold, the door can be determined to be stationary; 0074] during a specific time [where one of ordinary skill in the art would understand velocity and acceleration are measured with respect to a duration of time, where when the door is stationary, a start command can be sent to the camera device; when the door is closed, a stop command can be sent to the camera device; 0075];
based on detecting opening of the door, controlling the camera to capture an image of at least a part of an inside of the body [where step 101 is to obtain the first image of the specified space captured in the current shot; 0058; where when the door is stationary, a start command can be sent to the camera device; when the door is closed, a stop command can be sent to the camera device; 0075];
identifying, based on the template image [the second image; the image from the previous capture; 0077; 0086] and the image captured by the camera [a first image of a specified space currently captured; 0086], whether at least one of a first area of the inside of the body included in the image satisfies a predetermined condition [where if the number of pixel value is greater than or equal to a preset pixel threshold it is determined that the specified space has not changed and if the number of pixels with the same pixel value is less than the pixel threshold, it is determine the specified space has changed; 0082]; and
based on identifying that the predetermined condition is satisfied, storing the image as an optimal image [where if the specified space has changed the first image is uploaded to the server to be processed and to update the processing results to the monitoring platform; 0085; 0086].
Zhao does not teach the door comprising a door bin where the method comprises controlling the camera to capture an image of the door bin and storing the image as an optimal door bin image.
However, Takeshi teaches a method of processing image data in a refrigerator [0001] including the door comprising a door bin [door storage section 11a and 11b, Figure 11;0038] controlling the camera [image pickup device 30, Figure 1;0030] to capture an image of the door [where the image captured captures the storage compartment and the entire door storage section, Figure 11; 0035] and storing the image as an optimal door bin image [where after the SoC 41 performs image output processing on the processed image and the image is updated; 0029] where one of ordinary skill in the art would have been capable of applying this known technique, capturing door storage with a wide-angle fisheye lens, to a known device, a refrigerator, that was ready for improvement and the results would have been predictable to one of ordinary skill in the art i.e., allowing the entire interior of the fridge to be viewed [Takeshi; 0018].
Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the invention to modify the assembly of Zhao to have where the door comprising a door bin where the method comprises controlling the camera to capture an image of the door and storing the image as an optimal door bin image in view of the teachings of Takeshi where this known technique could have been applied to a known device that was ready for improvement and the results would have been predictable i.e., allowing the entire interior of the fridge to be viewed [Takeshi; 0018].
Regarding Claim 17, Zhao, as modified, teaches the invention of claim 16 and does not teach where the camera is on an upper end of the body, wherein the first area of the inside of the body is an area on a lower end and a side surface of the body covered by the door in a state in which the door is closed, and wherein the first line is on an upper end of the door.
However, Takeshi teaches a method of processing image data in a refrigerator [0001] where the camera is on an upper end of the body [image pickup device 30, Figure 1;0030], wherein the first area [storage compartment 11c, Figure 11; 0038] of the inside of the body is an area on a lower end and a side surface of the body covered by the door in a state in which the door is closed [a side of refrigerator 100 with box body 10, Figure 1 and annotated Figure 11], and wherein the first line is on an upper end of the door [first line of door storage section 11b, annotated Figure 11, where a door inherently includes lines to form its geometry] where one of ordinary skill in the art would have been capable of applying this known technique, a wide-angle fisheye lens, to a known device that was ready for improvement and the results would have been predictable to one of ordinary skill in the art i.e., allowing the entire interior of the fridge to be viewed with a single camera [Takeshi; 0018].
Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the invention to modify the assembly of the combined teachings to have where the camera is on an upper end of the body, wherein the first area of the inside of the body is an area on a lower end and a side surface of the body covered by the door in a state in which the door is closed, and wherein the first line is on an upper end of the door in view of the teachings of Takeshi where this known technique could have been applied to a known device that was ready for improvement and the results would have been predictable i.e., allowing the entire interior of the fridge to be viewed with a single camera [Takeshi; 0018].
Regarding Claim 20, Zhao, as modified, teaches the invention of claim 16 and further teaches wherein the identifying whether the predetermined condition is satisfied comprises: based on determining that a difference value between the first area included in the template image and the first area included in the image captured by the camera is a first predetermined value, determining that the predetermined condition is satisfied [ where the first image and the disturbed image can be compared pixel by pixel; 0082; where if the number of pixel value is greater than or equal to a preset pixel threshold it is determined that the specified space has not changed and if the number of pixels with the same pixel value is less than the pixel threshold, it is determine the specified space has changed; 0082].
PNG
media_image1.png
518
610
media_image1.png
Greyscale
Claims 6, 7 and 10 are rejected under 35 U.S.C. 103 as being unpatentable over Zhao (CN105472337A) and Takeshi et al. (WO2022038799A1) as applied to claim 1 and claim 9 above and in further view of Kim et al. (US20230043011A1).
Regarding Claim 6, Zhao, as modified, teaches the invention of claim 1 and further teaches a display [a terminal where execution entities of steps 101 to 103 can be applications located on the local terminal; 0064; where terminals include mobile phones, personal digital assistants (PDAs), tablet computers, and personal computers; 0055], but does not teach wherein the one or more instruction, when executed by the at least one processor, further cause the refrigerator to: based on receiving a user input to output the optimal door bin image, control the display output device to output the optimal door bin image.
However, Kim teaches an artificial intelligence refrigerator which is capable of photographing the inside of a refrigerator to recognize foods based on a photographed image [0001] where the one or more instruction [where one or more instructions constitute the learning model stored in memory 230; where AI device 100 may infer the result value for the input data using the learning model and generate the response or control command based on the result; 0079;0080], when executed by the at least one processor [processor 260 of AI server 200 and processor 180 of AI device 100, Figure 2 and Figure 4], further cause the refrigerator to: based on receiving a user input [via user input unit 123; 0099] to output the optimal door bin image [0305], control the display output device to output the optimal door bin image [where the image display area 1912 may be an area for displaying images corresponding to areas selected by the user among storage identification areas; 0305-0308] where one of ordinary skill in the art would have been capable of applying this known technique to a known device that was ready for improvement and the results would have been predictable to one of ordinary skill in the art i.e., improving convenience for the user by displaying contents of the refrigerator [Kim; 0005].
Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the invention to modify the assembly of the combined teachings to have where the one or more instruction, when executed by the at least one processor, further cause the refrigerator to: based on receiving a user input to output the optimal door bin image, control the display output device to output the optimal door bin image in view of the teachings of Kim where this known technique could have been applied to a known device that was ready for improvement and the results would have been predictable i.e., improving convenience for the user by displaying contents of the refrigerator [Kim; 0005].
Regarding Claim 7, Zhao, as modified, teaches the invention of claim 6, and further teaches wherein the one or more instructions, when executed by the at least one processor [a processor; 0109], further cause the refrigerator to: based on detecting the first door is opened, control the camera to obtain an optimal door bin image corresponding to the one of the first door [where the camera device can use an image sensor to capture surveillance images of the designated space; 0071; where when the door is stationary, a start command can be sent to the camera device; when the door is closed, a stop command can be sent to the camera device; 0075; refer to the rejection of claim 1 above] but does not teach where the door further comprises a first door and a second door, and wherein the one or more instructions, when executed by the at least one processor, further cause the refrigerator to: control the display output device to output the optimal door bin image corresponding to the one of the first door or the second door.
However, Takeshi teaches a method of processing image data in a refrigerator [0001] where the door further comprises a first door and a second door [refrigerator room doors 1a and 1b, Figure 1], and wherein the one or more instructions [where control SoC 41 issues a shooting instruction to camera 33; 0023], when executed by the at least one processor [where the control SoC41 (System on a chip) is an integrated circuit that integrates functions such as a processor core and other functions; 0016], further cause the refrigerator to: control the display output device [where the server 54 transmits fridge interior image data in response to a request to display a fridge interior image from a mobile device such as a smartphone 55 or a personal computer owned by the use, Figure 7; 0024] to output the optimal door bin image corresponding to the one of the first door or the second door [where multiple storage areas that are separated from each other in real life, such as a door storage section and a storeroom, or two door storage sections, are output as a single image, Figure 14; 0046] where one of ordinary skill in the art would have been capable of applying this known technique to a known device that was ready for improvement and the results would have been predictable to one of ordinary skill in the art i.e., allowing the entire interior of the fridge to be captured and viewed [Takeshi; 0046].
Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the invention to modify the assembly of the combined teachings to have where the door further comprises a first door and a second door, and wherein the one or more instructions, when executed by the at least one processor, further cause the refrigerator to: control the display output device to output the optimal door bin image corresponding to the one of the first door or the second door in view of the teachings of Takeshi where this known technique could have been applied to a known device that was ready for improvement and the results would have been predictable i.e., allowing the entire interior of the fridge to be captured and viewed [Takeshi; 0046].
Regarding Claim 10, Zhao, as modified, teaches the invention of claim 9, and does not teach where the one or more instructions, when executed by the at least one processor, further cause the refrigerator to: obtain, based on the image of the area of interest, information on incoming groceries or outgoing groceries, and control the display to provide information on the incoming groceries or the outgoing groceries.
However, Kim teaches an artificial intelligence refrigerator which is capable of photographing the inside of a refrigerator to recognize foods based on a photographed image [0001] where the one or more instructions [where one or more instructions constitute the learning model stored in memory 230; where AI device 100 may infer the result value for the input data using the learning model and generate the response or control command based on the result; 0079;0080], when executed by the at least one processor [processor 260 of AI server 200 and processor 180 of AI device 100, Figure 2 and Figure 4], further cause the refrigerator to: obtain, based on the image of the area of interest, information on incoming groceries or outgoing groceries [where food from the image is detected using the object detection model;0244;0277-0279], and control the display to provide information on the incoming groceries or the outgoing groceries [where the processor 180 displays food management information through the display 151 based on the acquired food storage state (S913);0278;0282] where one of ordinary skill in the art would have been capable of applying this known technique to a known device that was ready for improvement and the results would have been predictable to one of ordinary skill in the art i.e., improving convenience for the user by providing information of the refrigerator [Kim; 0018].
Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the invention to modify the assembly of the combined teachings to have where the one or more instructions, when executed by the at least one processor, further cause the refrigerator to: obtain, based on the image of the area of interest, information on incoming groceries or outgoing groceries, and control the display to provide information on the incoming groceries or the outgoing groceries in view of the teachings of Kim where this known technique could have been applied to a known device that was ready for improvement and the results would have been predictable i.e., improving convenience for the user by providing information of the refrigerator [Kim; 0018].
Claims 3, 5, 8, 13, 15 and 18 are rejected under 35 U.S.C. 103 as being unpatentable over Zhao (CN105472337A) in view of Takeshi et al. (WO2022038799A1).as applied to claim 1, 11 and 16 above and in further view of Chung et al. (US20230375258A1).
Regarding Claim 3, Zhao, as modified, teaches the invention of claim 1 and further teaches wherein the one or more instructions [0109], when executed by the at least one processor [a processor; 0109], further cause the refrigerator to:
compare the template image [where a second image of the specified space taken in the previous capture can be further acquired; 0077] with the image captured by the camera [a first image of a specified space currently captured; 0086; where the first image and the disturbed image can be compared pixel by pixel; 0082], and identify, based on a result of the comparison of the template image with the image captured by the camera, whether the first area of the inside of the body satisfy the predetermined condition [where if the number of pixel value is greater than or equal to a preset pixel threshold it is determined that the specified space has not changed and if the number of pixels with the same pixel value is less than the pixel threshold, it is determine the specified space has changed; 0082].
Zhao does not teach where the instructions cause the refrigerator to identify, based on a result of the comparison of the template image with the image captured by the camera, the first line included in the door satisfy the predetermined condition.
However, Chung teaches systems and methods for detecting a position of a door [0001] where the instructions [where the processor executes non-transitory programming instructions stored in memory; 0031] cause the refrigerator [refrigerator 100, Figure 1] to identify, based on a result of the comparison of the template image [where it may be determined in advance where the fiducial point should be; 0061] with the image captured by the camera [where the method includes an image capture sequence for analysis, evaluation, or comparison; 0056], the first line included in the door [where a fiducial point on the door is identified; 0060; where a fiducial point 210 may be identifiable by a counter line formed by an outer visible edge of the corresponding object on the door 112; 0038] satisfy the predetermined condition [where the fiducial point is compared to an intended location; 0061] where one of ordinary skill in the art would have been capable of applying this known technique, identifying a line included in the door for analysis, to a known device that was ready for improvement and the results would have been predictable to one of ordinary skill in the art i.e., isolating important object regions by edge matching [Chung, 0040].
Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the invention to modify the assembly of the combined teachings to have where the instructions cause the refrigerator to identify, based on a result of the comparison of the template image with the image captured by the camera, the first line included in the door satisfy the predetermined condition. in view of the teachings of Chung where this known technique could have been applied to a known device that was ready for improvement and the results would have been predictable i.e., isolating important object regions by edge matching [Chung, 0040].
Regarding Claim 5, Zhao, as modified, teaches the invention of claim 3 and further teaches wherein the one or more instructions, when executed by the at least one processor [0109], further cause the refrigerator to: based on determining that a difference value between the first area included in the template image and the first area included in the image captured by the camera is a first predetermined value, identify that the predetermined condition is satisfied [where the first image and the disturbed image can be compared pixel by pixel; 0082; where if the number of pixel value is greater than or equal to a preset pixel threshold it is determined that the specified space has not changed and if the number of pixels with the same pixel value is less than the pixel threshold, it is determine the specified space has changed; 0082].
Regarding Claim 8, Zhao, as modified, teaches the invention of claim 1 and further teaches where the camera is a wide angle camera [refer to the rejection of claim 1 in view of Takeshi], and does not teach wherein the one or more instructions, when executed by the at least one processor, further cause the refrigerator to: correct an image captured by the wide angle camera by removing one or more objects detected in the image, and identify whether the first area of the inside of the body included in the corrected image.
However, Takeshi teaches a method of processing image data in a refrigerator [0001] wherein the one or more instructions when executed by the at least one processor [SoC 41; 0028], further cause the refrigerator to to correct an image [where the control SoC 41 performs a fridge interior image correction process;0028] captured by the wide angle camera [imaging device 30 with a fisheye lens 34; 0015] by removing one or more objects detected in the image [where the cropping process can remove at least a portion of the area that is not a food storage area from the input photographed image, such as an area that shows only the wall surface of the refrigerator or the floor outside the refrigerator, and it is also possible to crop out the portions containing the food storage area contained in each image data, Figure 11; 0044] and identify whether the first area of the inside of the body included in the corrected image [where the image can be automatically determined to be of which storage room the captured image is from by a detection unit installed on the door or the drawer; 0033], where one of ordinary skill in the art would have been capable of applying this known technique, image processing, to a known device that was ready for improvement and the results would have been predictable to one of ordinary skill in the art i.e., eliminating the need for complex and expensive drive devices and drive controls to change the shooting range [Takeshi; 0036].
Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the invention to modify the assembly of the combined teachings to have wherein the one or more instructions, when executed by the at least one processor, further cause the refrigerator to: correct an image captured by the wide angle camera by removing one or more objects detected in the image, and identify whether the first area of the inside of the body included in the corrected image. in view of the teachings of Takeshi where this known technique could have been applied to a known device that was ready for improvement and the results would have been predictable i.e., eliminating the need for complex and expensive drive devices and drive controls to change the shooting range [Takeshi; 0036].
Zhao as modified does not teach wherein the one or more instructions, when executed by the at least one processor, further cause the refrigerator to: identify whether the first line included in the door included in the corrected image satisfy the predetermined condition.
However, Chung teaches methods for detecting a position of the refrigerator door [0001] where one or more instructions [where the processor executes non-transitory programming instructions stored in memory; 0031], when executed by the at least one processor [controller 150; 0031], cause the refrigerator [refrigerator 100, Figure 1] to: identify whether the first line included in the door included in the image [where a fiducial point on the door is identified; 0060; where a fiducial point 210 may be identifiable by a counter line formed by an outer visible edge of the corresponding object on the door 112; 0038; ] satisfy the predetermined condition [where the fiducial point is compared to an intended location; 0061] where one of ordinary skill in the art would have been capable of applying this known technique, ensuring the refrigerator door is in the desired open position, to a known device that was ready for improvement and the results would have been predictable to one of ordinary skill in the art i.e., isolating important object regions by edge matching [Chung, 0040].
Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the invention to modify the assembly of the combined teachings to have when executed by the at least one processor, further cause the refrigerator to: the first line included in the door included in the corrected image satisfy the predetermined condition in view of the teachings of Chung where this known technique could have been applied to a known device that was ready for improvement and the results would have been predictable i.e., isolating important object regions by edge matching [Chung, 0040].
Regarding Claim 13, Zhao, as modified, teaches the method of claim 11 and further teaches wherein the identifying whether the image satisfies the predetermined condition comprises: comparing the template image [where a second image of the specified space taken in the previous capture can be further acquired; 0077] with the image captured by the camera [first image of a specified space currently captured; 0086; where the first image and the disturbed image can be compared pixel by pixel; 0082]; and identifying, based on a result of the comparing, whether the first area of the inside of the body [where if the number of pixel value is greater than or equal to a preset pixel threshold it is determined that the specified space has not changed and if the number of pixels with the same pixel value is less than the pixel threshold, it is determine the specified space has changed; 0082 ] satisfy the predetermined condition [where if the number of pixel value is greater than or equal to a preset pixel threshold it is determined that the specified space has not changed and if the number of pixels with the same pixel value is less than the pixel threshold, it is determine the specified space has changed; 0082] and does not teach identifying, based on a result of the comparing, whether the first line included in the door satisfies the predetermined condition.
However, Chung teaches systems and methods for detecting a position of a door [0001] where based on a result of the comparison of the template image [where it may be determined in advance where the fiducial point should be; 0061] with the image captured by the camera [where the method includes an image capture sequence for analysis, evaluation, or comparison; 0056], the first line included in the door [where a fiducial point on the door is identified; 0060; where a fiducial point 210 may be identifiable by a counter line formed by an outer visible edge of the corresponding object on the door 112; 0038] satisfy the predetermined condition [where the fiducial point is compared to an intended location; 0061] where one of ordinary skill in the art would have been capable of applying this known technique, identifying a line included in the door for analysis, to a known device that was ready for improvement and the results would have been predictable to one of ordinary skill in the art i.e., isolating important object regions by edge matching [Chung, 0040].
Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the invention to modify the assembly of the combined teachings to have identifying, based on a result of the comparing, whether the first line included in the door satisfies the predetermined condition in view of the teachings of Chung where this known technique could have been applied to a known device that was ready for improvement and the results would have been predictable i.e., isolating important object regions by edge matching [Chung, 0040].
Regarding Claim 15, Zhao, as modified, teaches the invention of claim 13 and further teaches where the identifying whether the predetermined condition is satisfied comprises: based on determining that a difference value between the first area included in the template image and the first area included in the image captured by the camera is a first predetermined value, identify that the predetermined condition is satisfied [ where the first image and the disturbed image can be compared pixel by pixel; 0082; where if the number of pixel value is greater than or equal to a preset pixel threshold it is determined that the specified space has not changed and if the number of pixels with the same pixel value is less than the pixel threshold, it is determine the specified space has changed; 0082].
Regarding Claim 18, Zhao, as modified, teaches the invention of claim 16 and further teaches wherein the identifying whether the image satisfies the predetermined condition comprises: comparing the template image [where a second image of the specified space taken in the previous capture can be further acquired; 0077] with the image captured by the camera [first image of a specified space currently captured; 0086; where the first image and the disturbed image can be compared pixel by pixel; 0082]; and identifying, based on a result comparing whether the first area of the inside of the body [where if the number of pixel value is greater than or equal to a preset pixel threshold it is determined that the specified space has not changed and if the number of pixels with the same pixel value is less than the pixel threshold, it is determine the specified space has changed; 0082] satisfy the predetermined condition [where if the number of pixel value is greater than or equal to a preset pixel threshold it is determined that the specified space has not changed and if the number of pixels with the same pixel value is less than the pixel threshold, it is determine the specified space has changed; 0082] but does not teach identifying, based on a result of the comparison of the template image captured by the camera, whether the first line included in the door satisfies the predetermined condition.
However, Chung teaches systems and methods for detecting a position of a door [0001] including identifying, based on a result of the comparison of the template image [where it may be determined in advance where the fiducial point should be; 0061] with the image captured by the camera [where the method includes an image capture sequence for analysis, evaluation, or comparison; 0056], the first line included in the door [where a fiducial point on the door is identified; 0060; where a fiducial point 210 may be identifiable by a counter line formed by an outer visible edge of the corresponding object on the door 112; 0038] satisfy the predetermined condition [where the fiducial point is compared to an intended location; 0061] where one of ordinary skill in the art would have been capable of applying this known technique, identifying a line included in the door for analysis, to a known device that was ready for improvement and the results would have been predictable to one of ordinary skill in the art i.e., isolating important object regions by edge matching [Chung, 0040].
Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the invention to modify the assembly of the combined teachings to have identify based on a result of the comparison of the template image captured by the camera, whether the first line included in the door satisfies the predetermined condition in view of the teachings of Chung where this known technique could have been applied to a known device that was ready for improvement and the results would have been predictable i.e., isolating important object regions by edge matching [Chung, 0040].
Claims 4 and 14 are rejected under 35 U.S.C. 103 as being unpatentable over Zhao (CN105472337A) in view of Takeshi et al. (WO2022038799A1) and Chung et al. (US20230375258A1) as applied to claim 3 and 13 above and in further view of Guo et al. (CN114279367A).
Regarding Claim 4, Zhao, as modified, teaches the invention of claim 3 and further teaches wherein the one or more instructions [0109], when executed by the at least one processor [a processor; 0109], further cause the refrigerator to: detect a change of the first area of the inside of the body [where if the number of pixel value is greater than or equal to a preset pixel threshold it is determined that the specified space has not changed and if the number of pixels with the same pixel value is less than the pixel threshold, it is determine the specified space has changed; 0082] and the first line included in the door [refer to Chung as applied to claim 3 above] included in a plurality of images captured by the camera [a first image and a second image; 0082; refer to Chung as applied to claim 3 above], determine that the door is in a state of being open [where if the rotational angular velocity is positive and greater than or equal to the preset opening threshold, the door can be determined to be open; 0074] and halted [where after the door opens, if the acceleration is less than or equal to the preset stationary threshold, the door can be determined to be stationary; 0074], and based on determining that the door is in the state of being open and halted during the specific time [where one of ordinary skill in the art would understand velocity and acceleration are measured with respect to a duration of time, where when the door is stationary, a start command can be sent to the camera device; when the door is closed, a stop command can be sent to the camera device; 0075], obtain, as the template image [where the second image is an image from saved from a previous capture; 0077;0086], an image at a time point in the state in which the door is open and halted [where when the door is stationary, a start command can be sent to the camera device; when the door is closed, a stop command can be sent to the camera device; 0075; where if the specified space has changed the image is uploaded to the server to be processed and used to update the processing results to the monitoring platform; 0085; 0086].
Zhao does not teach the one or more instructions [0109], when executed by the at least one processor [a processor; 0109], further cause the refrigerator to determine, based on the plurality of images, whether the door is in a moving state, based on the change of the first area of the inside of the body being smaller than or equal to a predetermined value.
However Chung teaches systems and methods for detecting a position of a door [0001] to determine, based on the plurality of images [where the method 600 includes determining a shut state of the door based on the captured image(s) at 630; 0058], whether the door is in a moving state, based on the change of the first area of the inside of the body [where it may be determined that an opening of the door is likely; 0058], being smaller than or equal to a predetermined value [where if the measured brightness value is compared to the predetermined brightness threshold and determined to be greater than the predetermined brightness threshold, it may be determined that the door is in the partially opened state; 0062] where one of ordinary skill in the art would have been capable of applying this known technique, detecting a change in brightness value of a captured area, to a known device that was ready for improvement and the results would have been predictable to one of ordinary skill in the art i.e., improving the accuracy of the image processing algorithm by determining the state of the door without additional sensors [Chung, 0048].
Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the invention to modify the assembly of the combined teachings to determine, based on the plurality of images, whether the door is in a moving state, based on the change of the first area of the inside of the body being smaller than or equal to a predetermined value in view of the teachings of Chung where this known technique could have been applied to a known device that was ready for improvement and the results would have been predictable i.e., improving the accuracy of the image processing algorithm by determining the state of the door without additional sensors [Chung, 0048].
Zhao as modified further does not teach the one or more instructions [0109], when executed by the at least one processor [a processor; 0109], further cause the refrigerator to determine, based on the plurality of images, whether the door is in a moving state based on the first line included in the door while the door is open being smaller than or equal to a predetermined value.
However, Guo teaches a method for detecting the opening angle of a display case and a storage medium [n0001;n0002] including determining, based on the first line included in the door while the door is open [where the length of the bottom edge of the cabinet door is part of a triangle used to calculate the opening angle of the cabinet door, Figure 9; n0123-n0126] being equal to a predetermined value [where the target triangle is an equilateral triangle, such that the length of the bottom edge of the cabinet door is equal to the remaining side, Figure 9; n0126] where one of ordinary skill in the art would have been capable of applying this known technique to a known device that was ready for improvement and the results would have been predictable to one of ordinary skill in the art i.e., providing technical support for realizing other functions of the display cabinet based on the opening angle without additional sensors [Guo; n0031]
Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the invention to modify the assembly of the combined teachings to have the instructions cause the refrigerator to: based on the first line included in the door while the door is open being smaller than or equal to a predetermined value, determine that the door is in a state of being open and halted in view of the teachings of Guo where this known technique could have been applied to a known device that was ready for improvement and the results would have been predictable i.e., providing technical support for realizing other functions of the display cabinet based on the opening angle without additional sensors [Guo; n0031].
Regarding Claim 14, Zhao, as modified, teaches the invention of claim 13 and further teaches wherein the obtaining the template image comprises: detecting a change of the first area of the inside of the body [where if the number of pixel value is greater than or equal to a preset pixel threshold it is determined that the specified space has not changed and if the number of pixels with the same pixel value is less than the pixel threshold, it is determine the specified space has changed; 0082] and the first line included in the door [refer to Chung as applied to claim 13 above] included in a plurality of images captured by the camera [a first image and a second image; 0082; refer to Chung as applied to claim 13 above]; and based on determining that the door is in the state of being completely open [where if the rotational angular velocity is positive and greater than or equal to the preset opening threshold, the door can be determined to be open; 0074] and halted [where after the door opens, if the acceleration is less than or equal to the preset stationary threshold, the door can be determined to be stationary; 0074] during the specific time [where one of ordinary skill in the art would understand velocity and acceleration are measured with respect to a duration of time, where when the door is stationary, a start command can be sent to the camera device; when the door is closed, a stop command can be sent to the camera device; 0075], obtaining, as the template image [where the second image is an image from saved from a previous capture; 0077;0086], an image at a time point in the state the door was completely open and halted [where when the door is stationary, a start command can be sent to the camera device; when the door is closed, a stop command can be sent to the camera device; 0075; where if the specified space has changed the image is uploaded to the server to be processed and used to update the processing results to the monitoring platform; 0085; 0086].
Zhao as modified does not teach determining, based on the plurality of images, whether the door is in a moving state; and based on the change of the first area of the inside of the body being smaller than or equal to a predetermined value, determining that the door is in a state of being open and halted.
However Chung teaches systems and methods for detecting a position of a door [0001] including determining, based on the plurality of images [where the method 600 includes determining a shut state of the door based on the captured image(s) at 630; 0058], whether the door is in a moving state [where it may be determined that an opening of the door is likely; 0058], and based on the change of the first area of the inside of the body being smaller than or equal to a predetermined value [where if the measured brightness value is compared to the predetermined brightness threshold and determined to be greater than the predetermined brightness threshold, it may be determined that the door is in the partially opened state; 0062] determining that the door is in a state of being open and halted [image capture sequence; 0057] where one of ordinary skill in the art would have been capable of applying this known technique, detecting a change in brightness value of a captured area, to a known device that was ready for improvement and the results would have been predictable to one of ordinary skill in the art i.e., improving the accuracy of the image processing algorithm by determining the state of the door without additional sensors [Chung, 0048].
Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the invention to modify the assembly of the combined teachings to determine, based on the plurality of images, whether the door is in a moving state, based on the change of the first area of the inside of the body being smaller than or equal to a predetermined value in view of the teachings of Chung where this known technique could have been applied to a known device that was ready for improvement and the results would have been predictable i.e., improving the accuracy of the image processing algorithm by determining the state of the door without additional sensors [Chung, 0048].
Zhao as modified further does not teach determining, based on the plurality of images, whether the door is in a moving state based on the first line included in the door while the door is open being smaller than or equal to a predetermined value.
However, Guo teaches a method for detecting the opening angle of a display case and a storage medium [n0001;n0002] including determining, based on the first line included in the door while the door is open [where the length of the bottom edge of the cabinet door is part of a triangle used to calculate the opening angle of the cabinet door, Figure 9; n0123-n0126] being equal to a predetermined value [where the target triangle is an equilateral triangle, such that the length of the bottom edge of the cabinet door is equal to the remaining side, Figure 9; n0126] determining that the door is in a state of being open and halted [where the cabinet door reaches the target triangle n0126; when the cabinet door opening angle has been opened to the maximum angle; n0131] where one of ordinary skill in the art would have been capable of applying this known technique to a known device that was ready for improvement and the results would have been predictable to one of ordinary skill in the art i.e., providing technical support for realizing other functions of the display cabinet based on the opening angle without additional sensors [Guo; n0031]
Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the invention to modify the assembly of the combined teachings to have the instructions cause the refrigerator to: based on the first line included in the door while the door is open being smaller than or equal to a predetermined value, determine that the door is in a state of being open and halted in view of the teachings of Guo where this known technique could have been applied to a known device that was ready for improvement and the results would have been predictable i.e., providing technical support for realizing other functions of the display cabinet based on the opening angle without additional sensors [Guo; n0031].
Claims 19 is rejected under 35 U.S.C. 103 as being unpatentable over Kim et al. (US20230043011A1) in view of Takeshi et al. (WO2022038799A1) and as applied to claim 16 above and in further view of Guo et al. (CN114279367A) and Chung et al. (US20230375258A1)
Regarding Claim 19, Zhao, as modified, teaches the invention of claim 16 and further teaches where the obtaining the template image [where a second image of the specified space taken in the previous capture can be further acquired; 0077] comprises: detecting a change of the first area [where if the number of pixel value is greater than or equal to a preset pixel threshold it is determined that the specified space has not changed and if the number of pixels with the same pixel value is less than the pixel threshold, it is determine the specified space has changed; 0082] of the inside of the body included in a plurality of images captured by the camera [a first image and a second image; 0082]; and
based on determining that the door is in the state of being completely open [where if the rotational angular velocity is positive and greater than or equal to the preset opening threshold, the door can be determined to be open; 0074] and halted [where after the door opens, if the acceleration is less than or equal to the preset stationary threshold, the door can be determined to be stationary; 0074] during the specific time [where one of ordinary skill in the art would understand velocity and acceleration are measured with respect to a duration of time, where when the door is stationary, a start command can be sent to the camera device; when the door is closed, a stop command can be sent to the camera device; 0075], obtaining, as the template image [where the second image is an image from saved from a previous capture; 0077;0086], an image at a time point in the state the door was completely open and halted [where when the door is stationary, a start command can be sent to the camera device; when the door is closed, a stop command can be sent to the camera device; 0075; where if the specified space has changed the image is uploaded to the server to be processed and used to update the processing results to the monitoring platform; 0085; 0086].
Zhao as modified, does not teach detecting a change of the first line included in the door included in a plurality of images captured by the camera; and based on the change of the first line included in the door while the door is open being smaller than or equal to a predetermined value, determining that the door is in a state of being open and halted.
However, Guo teaches a method for detecting the opening angle of a display case and a storage medium [n0001;n0002] including detecting a change of the first line included in the door included in a plurality of images captured by the camera [S103 determine position information of the bottom edge of the display cabinet;n0039; determining the target triangle through the position information of the bottom edge; where the target image can be taken at the bottom of the display case every 0.5 second; n0038] determining, based on the first line included in the door while the door is open [where the length of the bottom edge of the cabinet door is part of a triangle used to calculate the opening angle of the cabinet door, Figure 9; n0123-n0126] being equal to a predetermined value [where the target triangle is an equilateral triangle, such that the length of the bottom edge of the cabinet door is equal to the remaining side, Figure 9; n0126] determining that the door is in a state of being open and halted [where the cabinet door reaches the target triangle n0126; when the cabinet door opening angle has been opened to the maximum angle; n0131] where one of ordinary skill in the art would have been capable of applying this known technique to a known device that was ready for improvement and the results would have been predictable to one of ordinary skill in the art i.e., providing technical support for realizing other functions of the display cabinet based on the opening angle without additional sensors [Guo; n0031]
Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the invention to modify the assembly of the combined teachings to have detecting a change of the first line included in the door included in a plurality of images captured by the camera; and based on the change of the first line included in the door while the door is open being smaller than or equal to a predetermined value, determining that the door is in a state of being open and halted in view of the teachings of Guo where this known technique could have been applied to a known device that was ready for improvement and the results would have been predictable i.e., providing technical support for realizing other functions of the display cabinet based on the opening angle without additional sensors [Guo; n0031].
Zhao as modified does not teach determining , based on the plurality of images, whether the door is in a moving state; and based on the change of the first area of the inside of the body being smaller than or equal to a predetermined value determining that the door is in a state of being open and halted.
However Chung teaches systems and methods for detecting a position of a door [0001] including determining, based on the plurality of images [where the method 600 includes determining a shut state of the door based on the captured image(s) at 630; 0058], whether the door is in a moving state [where it may be determined that an opening of the door is likely; 0058], and based on the change of the first area of the inside of the body being smaller than or equal to a predetermined value [where if the measured brightness value is compared to the predetermined brightness threshold and determined to be greater than the predetermined brightness threshold, it may be determined that the door is in the partially opened state; 0062] determining that the door is in a state of being open and halted [image capture sequence; 0057] where one of ordinary skill in the art would have been capable of applying this known technique, detecting a change in brightness value of a captured area, to a known device that was ready for improvement and the results would have been predictable to one of ordinary skill in the art i.e., improving the accuracy of the image processing algorithm by determining the state of the door without additional sensors [Chung, 0048].
Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the invention to modify the assembly of the combined teachings to determine, based on the plurality of images, whether the door is in a moving state, based on the change of the first area of the inside of the body being smaller than or equal to a predetermined value in view of the teachings of Chung where this known technique could have been applied to a known device that was ready for improvement and the results would have been predictable i.e., improving the accuracy of the image processing algorithm by determining the state of the door without additional sensors [Chung, 0048].
Response to Arguments
Applicant’s arguments filed 05/26/2026 with respect to claims 1, 11 and 16 have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
Applicant's arguments filed 05/26/2026 regarding the rejection of claim 8 under 35 U.S.C 103 have been fully considered but they are not persuasive.
On pages 16-17, Applicant argues regarding amended dependent claim 8 that the prior art Takeshi does not teach correcting an image captured by the wide angle camera by removing one or more objects in the image. Applicant's arguments have been fully considered but they are not persuasive. In particular, Takeshi teaches removing one or more objects detected in the image captured by the wide angle camera, 0033, where the cropping process can remove at least a portion of the area that is not a food storage area from the input photographed image, such as an area that shows only the wall surface of the refrigerator or the floor outside the refrigerator, and it is also possible to crop out the portions containing the food storage area contained in each image data, 0044. Refer to the rejection of Claim 8 above. Cropping and compositing achieves the result to remove one or more objects detected in an image. Accordingly, the rejections of record are considered proper and remain.
Applicant does not separately argue the rejection of claims 2-7, 9, 10, 12-15 and 17-20 except for their dependence upon claims 1, 11 and 16. Accordingly, the rejections of record are considered proper and remain.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to KEONA LAUREN BANKS whose telephone number is (571)270-0426. The examiner can normally be reached Mon-Fri 8:30- 5:00 EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Jerry-Daryl Fletcher can be reached at 5712705054. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/KEONA LAUREN BANKS/Examiner, Art Unit 3763
/ELIZABETH J MARTIN/Primary Examiner, Art Unit 3763
Prosecution Insights